Kockatoos Shopify MCP Server

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.0.9
Disambiguation2/5
Many tools have overlapping purposes, particularly around discounts (automatic vs code, basic/BXGY/free shipping) and legacy price_rule tools. Agents may struggle to choose the correct tool among 6 create discount tools, leading to misselection.
Naming Consistency4/5
Most tools follow a consistent verb_noun pattern (create, list, get, update, delete). However, legacy tools like create_price_rule break consistency with the newer discount tools, and 'stage_upload' is a different pattern.
Tool Count1/5
136 tools is far too many for a coherent server. This granularity exposes nearly every API endpoint, overwhelming agents. Typical well-scoped servers have 3-15 tools; this server has over 10 times that.
Completeness4/5
The tool set covers a wide range of Shopify resources (products, orders, customers, discounts, themes, files, etc.) with full CRUD for most. Minor gaps exist (no explicit refund tool, no bulk operations), but overall it's comprehensive.
Average 3.6/5 across 136 of 136 tools scored. Lowest: 2.2/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 0 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose side effects, permissions, or whether it is read-only. The 'get' implies read but lacks explicit confirmation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, no extra fluff. However, the misleading content reduces overall effectiveness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and a simple get operation, the description should be more accurate. The inconsistency makes it incomplete for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 100% coverage with a clear parameter description, but the main description introduces 'DiscountCodeNode ID' which conflicts with the schema's 'numeric Shopify price rule ID', adding confusion instead of value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description claims to get a code discount by DiscountCodeNode ID, but the tool is named 'get_price_rule' and takes a 'price_rule_id'. This contradiction between name and description confuses the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states it replaces older Price Rules workflow, but no guidance on when to use this vs sibling tools like get_code_discount.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It fails to disclose behavioral traits such as pagination (despite a 'page_info' parameter), rate limits, or that it reads data without side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is front-loaded and concise. However, it is too minimal to be fully helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description should explain what the tool returns. It does not. Given the complexity of discount codes and the presence of sibling tools, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no additional context about parameters beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the verb 'List' and the resource 'redeem codes', but it is vague by mentioning 'GraphQL code discount node' which is obscure. It does not clearly differentiate from sibling tools like 'list_code_discounts' or 'get_code_discount'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool versus alternatives. For example, it does not explain the relationship between this tool and 'list_code_discounts' or 'create_discount_code'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must convey behavioral traits. It only mentions that the tool uses the Admin GraphQL API and replaces an older workflow. There is no disclosure about permissions, idempotency, side effects, rate limits, or pagination behavior (despite pagination parameters in the schema).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary action. It is concise without redundant words, though it could include more detail without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks context about the return value, pagination handling, and how it differs from sibling list tools. Given the tool has 3 parameters and no output schema, the description should provide more guidance on expected behavior and usage patterns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, meaning all parameters have descriptions in the schema. The description does not add any additional meaning or examples beyond what the schema provides. Per guidelines, baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List code discounts using the Admin GraphQL API', but the tool name is 'list_price_rules', creating potential confusion between code discounts and price rules. The sibling tool 'list_code_discounts' exists, further blurring the distinction. While the verb 'list' is clear, the resource is ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'list_code_discounts' or 'list_automatic_discounts'. There is no mention of use cases, prerequisites, or when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must provide behavioral context. It only mentions 'Only provided fields are changed,' which implies a PATCH-like behavior, but lacks information on side effects, authorization requirements, error conditions, or whether the discount must be active.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences, but it sacrifices completeness for brevity. While it front-loads the core purpose, it omits necessary details, making it under-informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (16 parameters, nested objects, no output schema, no annotations), the description is far from complete. It fails to explain return values, error handling, or parameter dependencies, leaving the agent with insufficient context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 13% (2 of 16 parameters have descriptions). The description adds no parameter details, leaving the agent to infer from names alone. For a tool with many parameters, this is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates an existing automatic BXGY discount, with the specific verb 'Update' and explicit resource. The phrase 'Only provided fields are changed' clarifies the partial update semantics, distinguishing it from create tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus other update discount tools (e.g., update_automatic_discount_basic) or when not to use it. Prerequisites or alternatives are not mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits (e.g., idempotency, validation, side effects). An agent cannot infer important details like whether this tool can overwrite an existing code or what happens if the code is already in use.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words. It is appropriately front-loaded and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description is incomplete. It does not explain the return value, error conditions, or prerequisite states (e.g., that the price_rule_id must exist). The tool is simple but still leaves important gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% parameter coverage with clear descriptions. The description adds no additional meaning beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Set the customer-facing code') and the resource ('a basic GraphQL code discount node'), which distinguishes it from siblings like 'create_code_discount_basic' that likely create the entire discount. However, the verb 'set' could be more explicit about creation versus update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'create_code_discount_basic' or 'update_price_rule'. The description does not help the agent decide which tool to invoke in context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states 'Only provided fields are changed,' which discloses partial update (PATCH) behavior. However, no annotations are present, and the description does not cover permissions, idempotency, side effects, or whether certain parameters require simultaneous updates. The partial update note is useful but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (two sentences) and front-loaded with the core purpose. It is concise but could be more informative without sacrificing brevity. The partial update behavior is stated in the second sentence, which is appropriate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 17 parameters and no output schema, the description is insufficient. It lacks details on return values, error handling, required permissions, or the effect of omitting fields. The description only covers the basic operation and partial update aspect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no parameter-specific information beyond what the input schema provides. With only 18% schema description coverage, the description should compensate but does not mention any parameter details, formats, or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates an existing basic (amount-off) code discount. The verb 'update' and resource 'basic code discount' are specific. The parenthetical '(amount-off)' helps distinguish from other discount types, though it does not explicitly differentiate from sibling tools like update_code_discount_bxgy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as update_code_discount_bxgy or update_code_discount_free_shipping. There are no prerequisites, exclusions, or context about when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Only states 'Only provided fields are changed' (partial update), but no annotations exist; missing details on idempotency, error behavior, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise (two sentences), no wasted words; but could be better structured with additional context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 18 parameters, no output schema, and minimal description, the tool is highly incomplete for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 11% and description adds no parameter-specific meaning; does not compensate for low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Update an existing BXGY code discount', specifying both action and resource type, distinguishing it from siblings like update_code_discount_basic or update_code_discount_free_shipping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives; no mention of prerequisites, context, or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description indicates it's a mutation ('update') with partial updates, but lacks details on permissions, reversibility, or side effects. Minimal transparency for a 14-parameter tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence (10 words) that efficiently states the tool's purpose and partial update behavior. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 14 parameters, no output schema, and no annotations, the description is insufficient. It doesn't explain return values, prerequisites, or side effects. Completeness is lacking for a complex update tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no parameter-level information beyond the schema. With only 14% schema description coverage, the burden falls on the description to clarify parameter meaning, but it only states 'only provided fields are changed,' which is behavioral, not semantic.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing free shipping code discount' with a specific verb and resource. It differentiates from creation tools, but does not explicitly distinguish from other update discount tools like update_code_discount_basic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like update_code_discount_basic or update_automatic_discount_free_shipping. The only hint is the partial update behavior, which is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must convey behavioral traits. It mentions the required access scope but omits whether deletion is permanent, whether it affects live themes, or what happens if files don't exist. This is insufficient for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely short (one sentence), but lacks structure and is too terse. It communicates the core purpose but could be more efficient or add structure like separation of action and prerequisites.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a deletion tool with no output schema, the description should at least hint at return values or error handling. It only covers basic action and access scope, leaving out behavioral and contextual details crucial for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions fully cover both parameters ('theme_id' and 'filenames'), providing examples and clearly defining each. The description adds no further parameter details, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delete'), the object ('one or more files from a theme'), and the method ('by filename'). It distinguishes itself from siblings like 'delete_theme' (which deletes entire themes) and 'delete_files' (which deletes general files).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like 'delete_files' or 'upsert_theme_files'. The only contextual hint is the required access scope, but it doesn't explain scenarios or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses automatic application without code and criteria-based eligibility, but no annotations exist. Does not address combining discounts, overriding behavior, or side effects of setting destinationAll vs. countryCodes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and front-loaded with key info. Could be slightly longer to include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 10 parameters, no output schema, and no annotations, the description is too brief. Lacks explanation of response, required criteria logic, and limitations like maximum one free shipping discount per order.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60% (6/10 params have descriptions). The description adds no additional parameter meaning; does not explain how criteria like minimumSubtotal and destinationAll interact or whether they are exclusive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'create' and resource 'automatic free shipping discount'. Distinguishes from siblings like create_automatic_discount_basic (not free shipping) and create_code_discount_free_shipping (requires code).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like create_automatic_discount_basic or create_code_discount_free_shipping. Missing context about prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states creation. Does not disclose side effects, idempotency, error conditions, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with clear purpose. Very concise, but could benefit from slight expansion for completeness without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and 13 parameters; description does not explain return values, error handling, or how the discount interacts with other settings. Incomplete for a tool of this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 69% description coverage; description adds no extra meaning beyond schema. Baseline 3 is appropriate as schema does most of the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a free shipping code discount and mentions the customer code entry. It distinguishes from sibling tools like create_automatic_discount_free_shipping by specifying 'code' discount.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives (e.g., automatic discounts or other code discount types). No mention of prerequisites or typical scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. It only states the deletion action without mentioning side effects, permissions, reversibility, or response behavior, leaving significant gaps for a delete operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, concise, and front-loaded with the action. Every word serves a purpose: the first sentence states what it does, the second explains its legacy role. No unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 parameter, no output schema, no nested objects), the description is minimally adequate. However, it fails to clarify the ID type mismatch, leaves out return value or error handling, and omits any prerequisites for deletion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While the input schema describes 'price_rule_id' with 100% coverage, the description mentions 'by DiscountCodeNode ID', creating confusion about which ID type is expected. The description adds no value and introduces ambiguity, lowering the score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Delete a code discount by DiscountCodeNode ID', which is a specific action. However, the tool name and parameter 'price_rule_id' suggest a price rule, creating slight confusion between legacy and modern naming. It distinguishes itself as a compatibility wrapper among sibling delete tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description only mentions usability for compatibility via 'Preserves the legacy price-rule tool name for compatibility.' It does not explicitly state when to use this tool versus alternatives like delete_code_discount, nor provides any when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must disclose behavioral traits. It fails to mention that the operation is read-only, that results are paginated, or any constraints beyond what the schema shows. The description only states the action without behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, directly stating the purpose and a brief context. It is front-loaded and free of fluff, though the second sentence could be considered extraneous for tool selection.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks explanation of pagination, filtering semantics, and output format. Despite the schema providing parameter details, the agent needs more context on how to use the tool effectively, especially given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All four parameters have descriptions in the schema, so the baseline is 3. The description adds no extra meaning or context about parameters, so it does not improve upon the schema's coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'draft orders in the store', making the purpose unambiguous. It differentiates from similar tools like 'list_orders' and 'get_draft_order' by the resource type, but does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'get_draft_order' for a single item or 'list_orders' for regular orders. The second sentence provides background but no usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states the purpose (listing) and the nature of collections (automatic/rule-based). It does not mention side effects, authentication requirements, rate limits, or pagination behavior beyond schema parameters. The description adds little beyond the basic function.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences, with no redundant information. It efficiently conveys the core purpose. However, it could be slightly expanded with minimal additions to improve completeness without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters, no output schema, and no annotations, the description is incomplete. It does not explain return format, pagination details (beyond schema), common use cases, or how results are ordered. For a listing tool, this context is essential for correct agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all parameters with descriptions, achieving 100% coverage. The description does not add any additional meaning or context beyond the schema. Thus, the baseline score of 3 is appropriate; no extra value is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists smart (automatic) collections and explains that products are included based on rules. This distinguishes it from custom collections. However, it does not explicitly differentiate from other list tools like list_collections or get_smart_collection for single retrieval, which would have improved clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_smart_collection for a single collection or list_custom_collections for manual collections. The description does not specify prerequisites, limitations, or recommended contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behavior. It states 'Only provided fields are changed', implying partial update, but lacks details on idempotency, permission requirements, or effects on active discounts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, but it omits necessary context for a complex tool with 11 parameters. It is not verbose, but brevity comes at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (11 parameters, no output schema, low schema coverage) and the existence of sibling tools, the description is too minimal. It does not cover return values, errors, or how it differs from similar update tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 18%, with most parameters missing descriptions. The description adds that only provided fields change, which helps with update semantics, but does not explain individual parameters like destinationAll, maximumShippingPrice, or combinesWith.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool updates an existing automatic free shipping discount. The verb 'update' and resource 'automatic free shipping discount' are specific, and the description distinguishes it from sibling tools like update_automatic_discount_basic or create_automatic_discount_free_shipping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as update_automatic_discount_basic or create variants. No prerequisites, exclusions, or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'delete,' which implies a destructive action, but does not clarify if deletion is reversible, what happens to associated data, or the response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, making it concise, but it omits important details that could be included without significant verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema and the tool's destructive nature, the description is insufficient. It lacks details on success/failure, idempotency, or post-deletion behavior, which are crucial for a delete operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (webhook_id is fully described). The description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action (delete) and the resource (webhook subscription), distinguishing it from related tools like get_webhook, list_webhooks, and update_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool, prerequisites (e.g., webhook existence), or when not to use it (e.g., if webhook is still needed). Alternatives are not mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose side effects, but it does not. It does not mention whether adding a duplicate product is handled, if it overwrites, or if there are validation checks. Minimal behavioral information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is highly concise and front-loaded with the key action. However, it is arguably under-specified, but given the simple tool, it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation tool with no output schema and no annotations, the description should at least mention return values or side effects (e.g., no return, or success indicator). It only states the action, leaving the agent uninformed about behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover both parameters (product_id and collection_id) with clear, concise definitions. The description adds no further meaning, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add') and the objects ('product to a custom collection'). It distinguishes from sibling tools like 'remove_product_from_collection' and 'reorder_collection_products'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or conditions mentioned. Does not specify that the collection must be custom (though implied) or that the product must exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must fully disclose behavioral traits. It only states the action and required scope, omitting details about default behavior for duplicates, side effects, return format, or any limitations. This is insufficient for a write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, front-loading the main purpose. It avoids fluff, but could be slightly more informative without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, 2 enums, and no output schema or annotations, the description is notably incomplete. It misses explanation of when to use different content types, how duplicate resolution modes work, and what the result of creation looks like.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have schema descriptions (100% coverage), so the baseline is 3. The description repeats the source concept from 'original_source' but adds no new meaning for 'alt', 'content_type', or 'duplicate_resolution_mode' beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'create a file asset' and specifies the source types (external URL or staged upload URL) and supported file types. However, it does not explicitly distinguish from sibling tools like create_product_image or stage_upload, which share similar purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the required access scope ('write_files') and lists supported file types, providing some context. However, it lacks explicit guidance on when to use this tool versus alternatives, such as when to use stage_upload first or when create_product_image is more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It only states 'List all navigation menus' without mentioning that results are limited by the 'limit' parameter (default 50, max 250) or that it returns a list. No details on pagination or similar behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at 8 words, front-loading the core action. However, it is borderline too terse, omitting useful details that could be included without verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (1 parameter, no output schema), the description is incomplete. It claims to list 'all' menus but the schema imposes a limit (default 50). No explanation of what the response contains (e.g., array of menu objects).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter, so baseline is 3. The description does not add additional meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'navigation menus', distinguishing it from sibling tools like create_menu, delete_menu, and update_menu. The scope 'in the store' adds context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., get_menu for a single menu). No context on prerequisites or limitations beyond the schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that an invoice email is sent with a secure checkout link, but it does not mention side effects, state requirements (e.g., draft order must be in a certain state), idempotency, or response behavior. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence, concise and to the point. It front-loads the essential action. However, it could be slightly expanded with usage context without sacrificing conciseness. Thus, 4.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 5 parameters, no annotations, and no output schema, the description is too minimal. It lacks details on when to use, prerequisites, return behavior, and error conditions. A tool like this should provide more context for an AI agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with 5 parameters all described. The description adds 'secure checkout link' context but does not elaborate on parameter meaning beyond what the schema provides. Baseline 3 is appropriate as the schema handles parameter semantics adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Send an invoice email for a draft order with a secure checkout link.' It uses a specific verb ('Send') and resource ('draft order invoice') and distinguishes from siblings like 'complete_draft_order' by specifying the email sending aspect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use versus alternatives. There are related tools like 'complete_draft_order' that might also send notifications, but the description does not clarify when to choose this tool or indicate any prerequisites or constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must carry the full burden of behavioral disclosure. It only states it 'updates' and mentions legacy compatibility, but lacks details on side effects, permissions needed, rate limits, or what happens to the discount after update. This is insufficient for a mutation tool with no annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no unnecessary words. It front-loads the primary action and then explains the legacy context. Efficient, though could integrate more detail without much added length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite full schema parameter coverage, the description lacks completeness for a tool with 7 parameters and no output schema. It does not explain return values, any constraints (e.g., required fields beyond price_rule_id), or how the legacy compatibility affects behavior. The tool's relationship to the broader code discount system is also unclear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions, so the schema already provides parameter meaning. The description adds no additional parameter context beyond the generic 'modern basic code discount' label. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates a 'modern basic code discount' via GraphQL, which is a specific action. However, the tool name suggests 'price rule', and there is a sibling 'update_code_discount_basic' that appears to have overlapping purpose, creating confusion. The description identifies the resource but doesn't differentiate from similar tools sufficiently.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions preserving the legacy name for compatibility, hinting that this tool might be a transitional option. However, it fails to provide explicit guidance on when to use this tool versus alternatives like 'update_code_discount_basic' or when not to use it. The context is implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral details like irreversibility, side effects, or required order state. Only states 'completed and no longer active,' which is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, no filler, and the key action is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and a mutation tool (closing an order), the description lacks important context like idempotency, prerequisites (e.g., order must be open), and potential inventory implications.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The tool description adds no extra parameter meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('close') and resource ('order'), and explains the effect ('completed and no longer active'). It is specific and distinguishes from sibling tools like cancel_order.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., cancel_order). No mention of prerequisites or conditions for closing an order.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states it creates a discount. Does not disclose mutation, side effects, permissions, or whether it is destructive. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no redundancy. Front-loads purpose, then explains naming. All sentences are necessary and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 11 parameters and no output schema, the description is very brief. Does not explain workflow, required negative value, or relation to discount codes. Incomplete for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline 3. Description adds no parameter-level beyond the schema; no examples or interaction hints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it creates a modern basic code discount via GraphQL. It clarifies legacy naming but does not explicitly differentiate from sibling tools like create_code_discount_basic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies this is for legacy price-rule compatible discounts, but no explicit guidance on when to use this vs create_code_discount_basic or other discount tools. Lacks when-not or alternative references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the basic action. It does not disclose side effects, error behavior, return value, or processing delays (though variant_ids param mentions retry with update_product_image, that is not in the main description).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short, front-loaded sentences with no wasted words. It efficiently conveys the core purpose and optional behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no output schema, and no annotations, the description is too brief. It does not cover expected return, error conditions, or processing time. The variant retry note is only in param description, not main description, leaving gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all 5 parameters described in the schema). The description adds 'Add an image to a product by URL. Optionally assign it to specific variants,' which provides context but does not significantly enhance parameter understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add an image to a product by URL. Optionally assign it to specific variants.' It specifies the verb (Add), resource (image to a product), method (by URL), and optional variant assignment, distinguishing it from siblings like update_product_image and delete_product_image.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. The description does not mention prerequisites (e.g., product must exist, URL must be accessible) or contraindications (e.g., do not use if you need to update an existing image). Sibling tools exist but are not referenced.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It only states the action without disclosing whether the deletion is permanent, if it affects existing carts, or any other behavioral traits. The description is too minimal for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no unnecessary words. It is concise, though it could be more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (two required string parameters, no output schema), the description is adequate but not fully complete. It lacks details about what the deletion entails (e.g., permanent, impact on discounts), but for a straightforward delete, it provides the minimum viable information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters described in the schema. The description does not add any additional semantics or context beyond the schema field descriptions, so it meets the baseline but does not exceed it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'redeem code', and specifies the context 'from a GraphQL code discount node'. This distinguishes it from sibling tools like 'delete_code_discount' which likely deletes an entire discount, not individual codes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'delete_code_discount' or other delete tools. There is no mention of prerequisites, conditions, or scenarios where this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavior. It does not disclose what 'details' includes, whether it is read-only, required permissions, or error handling. The description is too minimal for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 7-word sentence. It is front-loaded with the verb and resource, and every word is necessary. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description does not explain what the returned 'details' include (e.g., image URL, dimensions). It also omits error scenarios. For a retrieval tool, this is insufficient for an agent to understand the response shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (both parameters have descriptions like 'The numeric Shopify product ID.'). The description adds no additional meaning beyond the schema, so it meets the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'details of a single product image'. It distinguishes itself from sibling tools like list_product_images (which lists multiple) and create/update/delete product image.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool vs alternatives, such as list_product_images to first find the image IDs, or that product_id and image_id are required. No exclusions or when-not-to-use stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It implies a read-only operation but does not disclose side effects, authentication needs, or rate limits. Minimal beyond purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, succinct, and front-loaded. Could include more details but efficient for a simple get operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple get with one param and no output schema, but missing information about return value (e.g., returns full webhook object). More context would improve agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and description adds no extra meaning. The schema already describes webhook_id as numeric. Baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (Get) and resource (details of a single webhook subscription), distinguishing it from siblings like list_webhooks, update_webhook, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, or when not to use it. Siblings exist (e.g., list_webhooks) but no differentiation provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description must disclose behavior. Only states 'List all metafields' but does not mention pagination (despite limit param), rate limits, auth needs, or error cases. Missing key behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single concise sentence. No fluff, but could be slightly longer to include important context like pagination. Still efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with well-documented params, description is adequate but missing behavioral context (pagination). No output schema, so return format is unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema provides 100% description coverage for all 3 parameters. Description adds no new semantic value beyond restating the purpose. Baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List', resource 'metafields', scope 'for an order', and method 'GraphQL'. Distinct from siblings like get_order_metafield (single) and list_product_metafields (different scope).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not mention pagination or that single metafield retrieval should use get_order_metafield. Leaves agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description lacks essential behavioral details. It does not mention pagination support, default limits, ordering, or any potential side effects, leaving the agent to infer from the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (6 words) with no wasted content. However, it could be slightly expanded to include pagination behavior without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation, the description is minimally adequate. It lacks mention of pagination (despite the page_info parameter) and does not describe the output format, which would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no additional context beyond what the schema already provides for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List all images') and the target resource ('for a product'). It accurately distinguishes from sibling tools like get_product_image (single image) and create_product_image.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For instance, it does not mention that get_product_image should be used for a single image or how pagination works.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Does not disclose pagination behavior or that results may be paginated despite parameters for pagination. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words. Could potentially include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple, but description lacks mention of return format or pagination. No output schema. Adequate for basic understanding but incomplete for full usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. Description adds no extra meaning beyond the schema. Baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'list', resource 'webhook subscriptions', and scope 'for this app'. Differentiates from siblings like get_webhook (singular) and create_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_webhook for a specific webhook. No context on prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It mentions 'upsert' implying create or update, but does not disclose side effects, error conditions, permission requirements, or behavior when order_id is invalid. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, very concise and front-loaded. Every word adds value, including the tech detail 'using metafieldsSet (upsert)'.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 5 parameters and no output schema or annotations, the description lacks crucial context: return value, error handling, constraints (e.g., type required for new), and aftereffects. Incomplete for effective agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description does not need to add parameter details. The description provides no additional parameter-level meaning beyond the schema, meeting the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates or updates an order metafield (upsert), specifying the verb and resource. It distinguishes from sibling tools like get_order_metafield or delete_order_metafield by indicating a write operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., set_product_metafield, delete_order_metafield). It does not mention prerequisites or context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds partial update behavior ('Only provided fields are changed') but lacks details on side effects, permissions, or error states. No annotations to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded action and key behavioral note. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, no annotations. Lacks return value info, error handling, or integration with related tools like add_product_to_collection. Incomplete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters. Description reiterates optionality but adds no new semantic detail beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action: update an existing custom collection. Distinguishes from sibling tools like create_custom_collection, delete_custom_collection, update_smart_collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., update_smart_collection, add_product_to_collection). No mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose behavioral traits such as idempotency, permission requirements, or whether the update is partial. The description adds minimal behavioral context beyond the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with a single sentence. No wasted words, but could include a bit more context without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple update tool with full schema coverage. However, it lacks usage guidelines and behavioral transparency. No output schema specified, leaving agent uncertain about response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and each parameter has a description. The tool description adds no additional meaning beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'tracking information for an existing fulfillment'. It distinguishes from sibling tools like create_fulfillment and cancel_fulfillment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites (e.g., fulfillment must exist). The description lacks explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose all behavioral traits. It mentions partial update ('Only provided fields are changed') but omits security, rate limits, error behavior, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that efficiently conveys purpose and partial update behavior. However, it could include more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderate-complexity tool with 4 parameters and no output schema or annotations, the description covers the partial update aspect but lacks information on return values, prerequisites, or edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters are described in the input schema (100% coverage). The description adds a general partial update behavior but does not enhance understanding of individual parameters beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates a smart collection's sort order or metadata, specifying the verb 'update' and the resource 'smart (automatic) collection'. It distinguishes from sibling tools like create or delete by focusing on updating existing collections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives like create_smart_collection or other update tools. The description lacks prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavior fully. It mentions restocking and status change but omits key details like reversibility, idempotency, permission requirements, or side effects on other resources.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, no wasted words. The action and effects are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description covers purpose and core effects. However, missing context about behavior when fulfillment is already cancelled and any constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds no extra meaning beyond 'The numeric fulfillment ID to cancel.' Baseline 3 applies as schema already documents the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'cancel' and resource 'fulfillment', and distinguishes from sibling 'cancel_order' by specifying the effects (restocks items, sets status). It is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool or alternatives. Does not mention prerequisites (e.g., fulfillment status must be not already cancelled) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose all behavioral traits. It only states it's an automatic amount-off discount. It does not mention side effects, prerequisites (e.g., requiring a cart), or that it creates a new resource. The description is too brief to adequately inform an agent about behavior beyond the obvious creation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks necessary detail. It could be considered under-specified rather than effectively concise. It earns its place but does not add value beyond the tool's name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (13 parameters, nested objects, no output schema), the description is severely lacking. It does not explain return values, confirmation of creation, or any criteria details. The agent would need additional context to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 13 parameters with 54% description coverage, but the description adds no parameter information. It does not explain any parameters or their roles. The agent must rely solely on the schema, which is incomplete. The description fails to compensate for the low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates an automatic amount-off discount (percentage or fixed). It explicitly says it's applied without a code, distinguishing it from code-based discounts. The term 'amount-off' and mentioning both percentage and fixed amount differentiates it from sibling tools like BXGY or free shipping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is for automatic discounts (no code required) and mentions 'when cart meets criteria,' indicating it's for condition-based discounts. However, it does not explicitly state when not to use it or contrast it with other automatic discount types (e.g., BXGY, free shipping). The context is clear but lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the only behavioral trait mentioned is 'applied without a code.' There is no disclosure of activation status, prerequisites, side effects, or how the discount interacts with existing ones. The description leaves many behavioral aspects unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, making it concise and front-loaded. However, this brevity sacrifices important detail that could be included without becoming overly long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (15 parameters, nested objects, no output schema), the description is insufficient. It fails to explain how buy/get quantities, product IDs, and collections work together, leaving significant gaps for an AI agent to correctly invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 47%, and the description adds no parameter-level detail. Parameters like endsAt, usesPerOrderLimit, buyAll, getAll lack clear meaning beyond their schema descriptions, and the description does not explain how they relate to the BXGY logic or how they are used together.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Create an automatic Buy X Get Y discount applied without a code.' It identifies the specific discount type (BXGY) and distinguishes it from other automatic discount types like basic or free shipping, listed among sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it (for BXGY automatic discounts) but does not provide explicit guidance on when not to use it or alternatives. The sibling tools list includes other discount creation tools, providing context, but the description itself lacks direct comparisons.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear full weight. It fails to disclose what happens after creation (e.g., if the draft order is immediately open, whether it returns an ID, or if further steps like completing are required). This is a significant gap for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with front-loaded purpose. Concise, but could be slightly more structured. Still, it is efficient and wastes no words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 8 parameters, nested objects, and no output schema, the description should mention the return value and lifecycle context. It lacks explanation of what is returned or how this fits with sibling tools like 'complete_draft_order'. It is minimally adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema covers all parameters 100%, the description inaccurately mentions 'billing addresses' while the schema only includes 'shipping_address'. This mismatch can mislead the agent. The description adds little beyond the schema for other parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Create a new draft order', clearly stating the verb and resource. It mentions key components like line items and optional customer, which distinguishes it from sibling tools like 'get_draft_order' or 'complete_draft_order'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives such as 'complete_draft_order' or 'update_draft_order'. It only states what to provide, not the context or lifecycle.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only states the action but does not disclose whether deletion is reversible, what happens to associated data, or any permissions required. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is concise and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation, the description minimally covers the purpose but lacks behavioral and usage context. It is adequate but not complete, especially given the large number of sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, both parameters have descriptions in the schema. The description adds no additional meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'image from a product', making it obvious what the tool does. It is distinct from sibling tools like delete_product or delete_variant.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There are many sibling delete tools (e.g., delete_product, delete_variant) but no context on prerequisites or idempotency.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states the action ('Get') but does not mention safety (read-only), error behavior, permissions, or rate limits. The 'by namespace and key' detail is already in the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter with full schema coverage and no output schema, the description is adequate but could mention the return value or expected outcome for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good descriptions for all three parameters. The description adds no new meaning beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that it gets a single customer metafield using namespace and key, which is a specific verb-resource combination that distinguishes it from sibling tools like list_customer_metafields or set_customer_metafield.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when to use get vs list, or any prerequisites. The description only states the action without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description implies read-only ('Get') but does not disclose potential side effects, authorization needs, or rate limits. Lacks detail beyond basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, efficient. Every word contributes to clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple retrieval tool with one parameter and no output schema. Description is adequate but could hint at return fields (e.g., inventory, pricing) for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter variant_id. Description does not add meaning beyond schema's own description, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Get', resource 'full details of a single product variant', and method 'by its ID'. Distinguishes from sibling list_variants and other get_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like get_product or list_variants. Context is implied but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It does not mention pagination (despite after cursor parameter), default limit, or that results are a list. No indication of permissions or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, well-structured sentence that efficiently states purpose and output. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
As a listing tool with pagination and filtering, the description omits pagination behavior and default limit. It provides output fields but not the format. Some gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so descriptions already document parameters. The tool description adds no extra meaning beyond confirming output fields (name, scopes, subscriptions). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists apps installed on the store and returns specific details (app name, scopes, subscriptions). It differentiates from siblings like list_products and the singular get_app_installation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., get_app_installation for a single installation). No mention of scenarios where filtering by category or sorting is beneficial.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It does not disclose read-only nature, rate limits, or pagination behavior beyond what the schema implies. Minimal behavioral context added.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with action and outcome. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple list tool with 3 parameters and no output schema. However, it lacks explanation of return format or pagination, which would be useful given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% parameter coverage with descriptions. Description adds no new parameter-specific details beyond confirming filtering capability. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists code discounts and specifies types (basic, BXGY, free shipping). The tool name and description distinguish it from automatic discount tools, but no explicit differentiation from 'list_discount_codes' sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for listing with filtering but does not explicitly state when to use this tool over alternatives like 'list_discount_codes' or 'get_code_discount'. Usage context is clear but lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only mentions 'via GraphQL' without explaining pagination, rate limits, or what data is returned (e.g., keys, values). This is insufficient for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loading the main action with no fluff. It is concise, though it could be slightly expanded without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and only 3 parameters, the description is too minimal. It fails to explain return format, pagination behavior, namespace filtering nuances, or the limit parameter's effect, leaving agents without sufficient context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and all parameters have descriptions. The description adds no extra meaning beyond what is already in the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), the resource (metafields for a customer), and the method (GraphQL). It distinguishes this tool from siblings like get_customer_metafield (single retrieval) and set/delete variants.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like get_customer_metafield. The description implies listing all, but does not mention filtering or when to choose this over related list tools (e.g., list_product_metafields).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions pagination and date filtering, which are important traits. However, it does not state that the tool is read-only, nor does it explain return format or pagination behavior (e.g., cursor handling). The description is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, no filler, and the main action is front-loaded. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters and no output schema, the description is adequate. It covers the essential features (filtering and pagination) but lacks specifics about return format or pagination details. It does not mention any edge cases or limitations, but for a simple listing tool, this is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, meaning all 5 parameters are documented in the input schema. The description adds no new parameter-level details beyond summarizing that it supports pagination and date filtering. It does not explain the interplay between parameters (e.g., since_id vs page_info). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'customers', and specifies 'in the store' as the scope. It also mentions supported features (pagination, date filtering), distinguishing it from other customer-related tools like search_customers. However, it could be more precise about what 'basic filtering' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives such as search_customers. No context about prerequisites, limitations, or comparative advantages is given, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist; description only restates the action. Does not disclose consequences (e.g., permanent removal, error states). Fails to add context beyond the name and schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and front-loaded. Efficient but could include minimal context without bloat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema or annotations. Description omits success/failure behavior, prerequisites, or side effects. Incomplete for a destructive action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. Tool description adds no further meaning, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action 'Remove' and resource 'product from a custom collection'. Distinguishes from siblings like 'add_product_to_collection' and 'reorder_collection_products'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives provided. Usage is implied but not differentiated from other removal methods like updating the collection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description discloses partial update behavior but lacks other behavioral traits such as prerequisites (discount must exist), idempotency, side effects, or error handling. Bar is higher due to no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, concise and front-loaded: first sentence states verb and object, second adds important nuance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite complexity (14 params, nested objects, no output schema), description is very brief. Missing details on behavior when discount not found, idempotency, return value, or required fields beyond 'id'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 14% (only 'id' described). Description adds no parameter explanations beyond 'Only provided fields are changed', failing to compensate for low coverage. Does not explain semantics of any parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Update an existing basic automatic discount', specifying verb (update) and resource (basic automatic discount). Differentiates from siblings like create_automatic_discount_basic and other update discount types (bxgy, free_shipping).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage context (updating existing discount with partial update via 'Only provided fields are changed') but provides no explicit guidance on when to use this vs other update discount tools or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must fully disclose behavior. It mentions converting to a real order and optionally marking as paid, but omits side effects (e.g., irreversibility, permissions, inventory impact, notifications).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short, front-loaded with the action, and contains no extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description lacks completeness. It does not specify return values, error conditions, or what happens to the draft order after completion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and adequately explains both parameters. The description adds minimal value beyond the schema, only providing context for the optional payment marking.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Complete a draft order'), the resource ('draft order'), and the outcome ('converting it into a real order'). It distinguishes from related tools like 'create_draft_order' and 'cancel_order'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (to finalize a draft order) but does not explicitly state when not to use or provide alternatives. It lacks prerequisites or constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It states the tool creates a BXGY discount, but does not disclose behavioral details like whether it overwrites existing codes, requires specific permissions, or has rate limits. The schema provides field-level detail, but behavioral context is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the tool's primary purpose (create BXGY code discount). It is efficient with no redundant words, though it could benefit from a brief note on when to use it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (17 parameters, nested objects, enums) and many siblings, the description is too brief. It does not explain the buy/get logic, interaction with other discounts, or provide examples. The schema covers field specifics, but overall context for selecting this tool is lacking.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 88%, so the schema already documents most parameters. The description does not add extra meaning beyond the general overview; it does not explain field relationships or provide examples. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create), the specific type of discount (Buy X Get Y code discount), and the mechanism (customers enter a code). It distinguishes itself from sibling tools like create_code_discount_basic and create_code_discount_free_shipping by explicitly naming 'BXGY'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., basic or free shipping discounts). It does not mention prerequisites, limitations, or scenarios where another discount type would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that product IDs are optional but does not mention any side effects, permissions, rate limits, or whether the operation is reversible. For a creation tool, this is insufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences that are front-loaded. It lacks wasted words, but could be slightly expanded to include key details without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given six parameters and no output schema, the description covers the core action and optional product IDs. However, it omits what the tool returns (e.g., the created collection object) and any constraints, which is a gap for a mutation tool with no annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds minimal value by hinting at the 'collects' parameter, but it does not elaborate on any other parameters (e.g., title, sort_order) beyond what is already in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Create a new custom (manual) collection,' clearly indicating the verb and resource. The term 'manual' differentiates it from create_smart_collection, a sibling tool, ensuring the agent can distinguish between them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by saying 'Optionally include product IDs to add to it,' which is a contextual hint. However, it does not explicitly state when to use this tool vs. alternatives (e.g., smart collections) or provide any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It mentions marking items as shipped and optional tracking, but does not disclose side effects like inventory changes, irreversibility, or notification behavior (though notify_customer parameter exists). Lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No extraneous words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, 1 required, and no output schema, the description is minimal. Misses prerequisites (e.g., fulfillment orders must be in a fulfillable state), return value format, or confirmation of success. Could be more helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (all parameters described). The description adds context about marking shipped and tracking info, but does not provide additional meaning beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a fulfillment, marks items as shipped, and optionally adds tracking info. It uses specific verbs and resource, and is distinct from siblings like 'cancel_fulfillment' or 'update_fulfillment_tracking'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (to create a fulfillment after orders are ready), but does not explicitly state when not to use or compare with alternatives like 'list_fulfillments' or 'update_fulfillment_tracking'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It does not confirm this is a read-only operation or disclose any side effects. For a listing tool, this is a notable gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (two sentences) and front-loads the core purpose. However, it contains inaccurate statements that could mislead, making it less efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks critical context such as what data the response contains (order fields) and how to use pagination parameters (since_id, page_info) correctly. Given no output schema, more detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While schema coverage is 100%, the description contains inaccuracies: it claims 'Use `status` to filter by fulfillment status' but status actually filters by order status (open/closed/cancelled). It also states 'By default returns open orders' but the schema default for status is 'any'. These errors reduce its helpfulness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List orders in the store' with a specific verb and resource. It distinguishes from sibling list tools by mentioning default behavior (open orders) and available filters, making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies when to use the tool by noting default returns open orders and suggesting status/financial_status filters. It also mentions pagination. However, it does not explicitly state when to avoid this tool or mention alternatives like get_order for detailed info.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only says 'activate' without disclosing side effects, impact on scheduled discounts, or any behavioral details beyond the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, front-loads the action and condition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple activation with one param, description is adequate but lacks information about return values or confirmation. No output schema, so agent may not know expectation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one parameter with description; schema coverage is 100%. Description adds no extra meaning beyond the schema, meeting baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'activate' and resource 'code discount', specifying condition 'currently inactive or scheduled'. Differentiates from sibling tools like deactivate_code_discount.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage from description but no explicit guidance on when to use vs alternatives like deactivate_code_discount or create_code_discount_*. No when-not or alternative mentions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that cancellation may trigger a refund based on restock/settings, but omits details like required permissions, side effects on order status, or whether the operation is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with core purpose, immediately followed by key optional parameters. No superfluous details; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description adequately covers the primary action and main side effect (refund). However, it lacks information on success/error responses, idempotency, or cancellation timing, making it moderately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds only contextual repetition ('optionally specify a reason and email') without explaining enum values or default behavior beyond what the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Cancel' and the resource 'an order', distinguishing from sibling tools like 'close_order' or 'delete_order'. It specifies key actions (reason, email) and a behavioral condition (refund trigger).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives (e.g., 'close_order', 'delete_draft_order'). It does not mention prerequisites, post-conditions, or exclusion scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It states the tool creates a definition at the store level and mentions validations, but it does not disclose whether the operation is idempotent, what happens if the definition already exists, required permissions, or any side effects. This is minimal disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long. The first sentence states the purpose, the second explains what definitions do, and the third provides a usage hint for validations. No unnecessary words or repetition, making it concise and front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, 3 required, no output schema, and no annotations, the description is adequate but not thorough. It covers the core purpose and validations but lacks prerequisites, permissions, idempotency, or return value hints. It is sufficient for a straightforward creation tool but leaves gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 8 parameters with 100% description coverage, so the schema already explains each parameter. The description adds brief extra context for validations ('e.g. allowed choices for text fields') and mentions the 'validations' parameter, but this does not significantly surpass the schema details. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new metafield definition for products at the store level.' It uses a specific verb ('create') and resource ('metafield definition'), and distinguishes from sibling tools like list_metafield_definitions or delete_product_metafield by focusing on creation and definition-level structure.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives context for validations ('Use `validations` to set constraints like allowed choices for text fields.'), implying when to use the tool (to define structure before assigning metafields). However, it does not explicitly state when not to use this tool or mention alternatives, so guidance is implied but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. The description only states the action and that it does not delete, but omits behavioral details such as side effects (e.g., impact on existing carts or orders), reversibility (implied 'pause'), or state changes. Minimal transparency for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, 10-word sentence that is front-loaded and efficient. Every word contributes to the understanding. No wasted content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and no output schema. The description is adequate for the core action but lacks context on the implications of deactivating (e.g., whether it can be reactivated, if it affects active discounts). Given sibling tools like activate_code_discount, more completion could be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter ('id') fully described as 'GID of the discount node.' The tool description does not add additional meaning beyond the schema—baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('deactivate/pause') and resource ('code discount'), and explicitly distinguishes from deletion ('without deleting it'). This clarifies the tool's purpose and differentiation from sibling tools like delete_code_discount.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (when you want to temporarily disable a discount) and not to use (when you want to permanently remove it), but lacks explicit guidance on when to deactivate vs. activate or delete. No alternatives or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only says 'Delete' without disclosing consequences (e.g., irreversibility, permission requirements). A delete operation should warn about permanence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no unnecessary words. Fully front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete with one parameter, the description is adequate in covering the core action. However, missing behavioral context (e.g., irreversibility) and no annotations reduce completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and already explains the parameter. The description reiterates the parameter meaning but adds no new details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (delete), resource (customer metafield), and identification method (by GID or numeric ID). It unambiguously distinguishes from sibling tools like get, list, and set metafield tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives, but the action is a standard delete operation; the purpose is self-explanatory. However, no exclusions or comparisons are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description mentions 'permanently delete' (destructive) and a constraint, but lacks details on irreversibility, required permissions, or side effects on related resources.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no extraneous information, efficiently conveying the purpose and a key limitation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 1-param tool, the description covers the basic purpose and a constraint, but could include more about irreversibility or error conditions for better completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description does not add significant meaning beyond what the schema already provides for the single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'permanently delete' and the resource 'draft order', distinguishing it from other delete tools and order management tools like cancel_order or complete_draft_order.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a key constraint ('Cannot delete completed draft orders') but does not explicitly guide when to use this tool over alternatives or suggest prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only mentions jsonValue serialization. Lacks disclosure about permissions, error handling, or data freshness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the primary purpose and add a technical detail. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with clear parameters, but no output schema. Missing description of return format (e.g., metafield object structure) which would help the agent process the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%; description adds no extra meaning beyond the schema. Baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (get), target (single metafield), identifiers (namespace and key for product), and method (GraphQL). Distinguishes from siblings like list_product_metafields and set_product_metafield.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for retrieving a specific metafield when namespace and key are known, but no explicit guidance on when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Only states basic function with no disclosure of read-only nature, rate limits, or side effects. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose. No wasted words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema. Description is adequate for a basic listing but lacks details about return format, pagination behavior (though schema covers page_info), and additional context that would help an agent fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% description coverage for all 3 parameters. Description adds no extra meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), the resource (products), and the filter (belonging to a specific collection). It distinguishes from sibling tools like list_products (unfiltered) and add_product_to_collection (adds, not lists).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. The description implies usage for getting products of a collection, but does not mention alternatives or context for choosing this over list_products.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'list', implying a read operation, but does not mention pagination behavior, sorting, rate limits, or any side effects. This is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action, and the second sentence provides essential context. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 5 parameters and no output schema, the description is minimal. It lacks details on return format, pagination, or ordering, which would improve completeness for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description. The tool description adds no extra meaning beyond the schema, meeting the baseline for this dimension.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List custom (manual) collections' and explains what custom collections are, distinguishing them from smart collections. The verb 'list' and resource 'custom collections' are specific, and the context helps differentiate from sibling tools like list_smart_collections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for manual collections but does not explicitly state when to use this tool versus alternatives like list_smart_collections. There is no guidance on exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries some burden. It reveals the required access scope (read_files) which is a behavioral trait, but lacks details on pagination behavior, rate limits, or default sorting. Transparency is average.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences. The first sentence defines the purpose, and the second highlights key features and requirements. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and no annotations, the description covers the basics (purpose, access scope, filtering) but does not explain return format or pagination behavior. It is somewhat incomplete for a tool with 5 parameters and no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description only adds a vague mention of filtering by 'media type, filename, status, and more,' which aligns with the query parameter. It does not add significant new meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (list) and resource (files uploaded to the store) and gives examples of file types. It is specific enough to distinguish from other list tools, though it does not explicitly contrast with siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions required access scope (read_files) and that filtering is supported, but it does not provide guidance on when to use this tool over alternatives or when not to use it. Usage is implied but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes a listing operation but fails to disclose read-only nature, error states, or requirements beyond the parameter. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the action, second defines the resource. No wasted words, efficient for quick understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple list tool with one parameter, but lacks output schema and does not mention pagination, empty results, or expected behavior for invalid order IDs. Could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the parameter is already well-documented (The numeric Shopify order ID). The description adds no additional meaning beyond what the schema provides, meeting the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List fulfillment orders for an order' with a specific verb and resource. The additional definition of fulfillment orders distinguishes it from sibling tools like list_fulfillments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage via order_id but provides no guidance on when to use this tool versus alternatives (e.g., list_fulfillments or get_order). No exclusion criteria or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Mentions required access scope and Shopify exemption, but does not disclose whether action is destructive, reversible, or what happens to the previous live theme. Adds some context beyond name but lacks completeness for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that is efficient and front-loaded with the core action. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description adequately covers purpose and access requirements. However, it omits details on the effect on the previous live theme and any return value, which is a gap given the significance of the action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes the sole parameter 'theme_id' with its format (GID). The description adds no additional meaning beyond the schema. With 100% schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Publish' and resource 'theme', and explains the effect 'making it the live storefront theme (MAIN role)'. Distinguishes from sibling tools like update_theme or delete_theme.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Does not specify when to use this tool versus alternatives like update_theme or create_theme. No guidance on prerequisites beyond access scopes, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the transparency burden. It reveals that it's a partial update ('only provided fields are changed'), but lacks details on permissions, side effects (e.g., does updating email trigger a customer update?), or state restrictions. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two short, direct sentences that are front-loaded with the key information. Every sentence serves a purpose, making it concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the basic action and partial update behavior, it omits important context such as error conditions (e.g., order not found), idempotency, and whether certain parameters require specific permissions. For a tool with 4 parameters and no output schema, this is a moderate gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides full descriptions for all 4 parameters (100% coverage). The description's mention of 'notes or tags' is redundant with the schema and adds no new semantic information. Therefore, the description does not enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'update' and the resource 'existing order', distinguishing it from sibling tools like cancel_order. It also specifies typical use cases (notes/tags) and the behavior of partial update, making the purpose very clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implicit usage by stating 'Typically used to add/edit notes or tags' and 'Only provided fields are changed', but it does not explicitly compare to alternatives or state when not to use it. There is no guidance about prerequisites or when to use other order update tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states 'update' but does not clarify partial vs full replacement, whether missing optional fields are retained, or what the return value is. Side effects of modifying variant_ids are also unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence front-loading the action and key details. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description is adequate but misses behavioral context (e.g., patch semantics, idempotency) and return value expectations. Given no output schema and no annotations, 'adequate' is appropriate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The description only echoes the schema's listed fields (alt, position, variant_ids) without adding meaning or usage constraints. Baseline 3 due to high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (update), resource (product image), and specific attributes (alt text, position, variant assignments). This distinguishes it from siblings like create, delete, get, and list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. However, the purpose is clear enough that an agent can infer it's for modifying an existing product image. Missing guidelines about prerequisites or alternatives among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only mentions creation and required scope, but fails to disclose other behavioral traits such as side effects, limits, or return behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence concisely conveys the purpose and a key requirement. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description lacks important context such as what is returned on success, error conditions, or details about the nested items structure beyond schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%. The description does not add meaning beyond what the schema already provides for the three required parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a new navigation menu for the online store.' It is a specific verb-resource combination that distinguishes from sibling tools like update_menu and delete_menu.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions the required access scope 'write_online_store_navigation', but does not explicitly state when to use this tool over alternatives. However, the context of creation is implicit and clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It states 'Delete', implying permanent removal, but provides no further context about consequences, permissions, or reversibility. For a simple deletion, this is adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the action, and contains no filler words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with one parameter and no output schema, the description is adequately complete. It misses no essential information, though it could briefly explain what an order metafield is.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% as the parameter is described. The description repeats the parameter info ('by its GID or numeric ID') but does not add new meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete an order metafield' which is a specific verb+resource combination. It differentiates from siblings like delete_product_metafield and delete_customer_metafield by specifying 'order'. The method 'by its GID or numeric ID' is also helpful.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. There is no mention of when to use this tool versus alternatives like set_order_metafield or get_order_metafield, nor when not to use it. The description simply states what the tool does.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It fails to disclose behavioral traits such as idempotency, error handling (e.g., 404 if not found), authentication requirements, or rate limits. Only the basic operation 'Get full details' is stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 18-word sentence that is front-loaded and to the point. Every word adds value with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-customer lookup with one required parameter and no output schema, the description covers the key return fields (addresses, order count, total spent). Missing error handling details, but overall adequate for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter (customer_id) already described as 'The numeric Shopify customer ID.' The description adds 'by their ID' but no new semantic detail beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'full details of a single customer', and the scope 'by their ID'. It also lists specific included fields (addresses, order count, total spent), distinguishing it from sibling tools like list_customers or search_customers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving detailed single-customer info but does not explicitly state when to use this vs alternatives like list_customers for overviews or search_customers for filtered queries. No when-not or alternative guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Only states 'Get full details' implying read-only, but lacks specifics on auth, rate limits, or what 'full details' entails. Minimal behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no filler. Front-loaded with verb and resource. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple (1 param, no output schema). Description states core purpose but could mention return value structure or that it returns the full draft order object. Slight gap compensated by low complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 100% with parameter description 'The numeric Shopify draft order ID.' Description adds no additional meaning beyond schema baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Get', resource 'draft order', and scope 'single by ID'. Distinguishes from sibling tools like list_draft_orders and other get_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage: use when you have a draft order ID and need details. No explicit when-not-to-use or alternatives provided, but context suggests basic fetch operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must carry the burden. It implies a read operation but does not explain error handling (e.g., missing ID), authorization needs, or rate limits. However, the behavior is straightforward for a get-by-id.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at 10 words, with no wasted information. It efficiently conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter getter with no output schema, the description adequately explains what the tool does. It could optionally mention the return format, but it is not necessary for basic understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%: the parameter location_id is already well-described in the schema. The description adds no additional meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'details of a single location by its ID'. It distinguishes itself from siblings like list_locations by specifying 'single'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as list_locations, nor any prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the tool returns the full nested item structure, which is useful behavioral context. However, it does not disclose whether the operation is read-only (likely), any potential side effects, or error conditions (e.g., handle not found). The description is minimal and lacks depth for a tool with no structured behavioral metadata.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no unnecessary words. It front-loads the purpose and includes the key behavioral detail (full nested item structure). Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-identifier tool with one parameter and no output schema, the description is nearly complete. It specifies the parameter (handle) and the return content (full nested item structure). Minor missing details: whether the tool returns null or an error if the menu is not found, and that the operation is read-only. However, given the simplicity, it is sufficiently informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of the parameter with a description for 'handle'. The tool description adds 'by its handle' which essentially reiterates the schema. No additional meaning or constraints beyond what the schema provides. With high schema coverage, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a navigation menu by its handle and includes the full nested item structure. This distinguishes it from list_menus (which lists all menus) and create/update/delete_menu. The verb 'get' plus resource 'menu' and qualifier 'by its handle' is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like list_menus or other get tools. The description implies usage when a specific menu handle is known and the full nested structure is needed, but it does not provide direct context or exclusions. The presence of sibling tools like list_menus implies this is for single menu retrieval, but the description does not state it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose behavioral traits like read-only nature, pagination behavior, or rate limits. It only mentions filtering capability, missing important safety context for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no unnecessary words. Highly concise and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description does not specify return format or pagination handling. Missing details about what the tool returns, which is important for a list operation. Incomplete given the absence of output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds 'filtering by status, type, and text search' which is already covered by the query parameter description in schema. No additional meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists automatic discounts (applied without a code) using GraphQL, distinguishing it from sibling list_code_discounts by specifying the discount type. The verb 'list' and resource 'automatic discounts' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly guides when to use by contrasting 'without a code' with code discounts, but does not explicitly state when not to use or mention alternatives. Clear context is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It discloses that the tool returns tracking info and status, but does not mention pagination behavior, authorization needs, or any side effects. It adequately describes the read-only nature implicitly but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that conveys the core purpose without any wasted words. It is front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 3 parameters with full schema descriptions, the description provides sufficient context about return content (tracking, status). It could mention pagination but that is covered in the schema. Overall complete for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions present. The description adds a brief hint about returned data (tracking info and status) but does not enhance understanding of parameters beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all fulfillments for a specific order, including tracking info and status. It uses specific verb 'list' and resource 'fulfillments for an order', distinguishing it from sibling tools like list_fulfillment_orders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., create_fulfillment, cancel_fulfillment, list_fulfillment_orders). The description lacks explicit context, exclusions, or references to sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must reveal behavioral traits. It mentions the required access scope, but fails to disclose that the items parameter completely replaces existing items, or any side effects. This is a significant gap for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it could be improved by incorporating key behavioral details such as the replacement behavior of items.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, no output schema, and no annotations, the description is minimal. It covers basic purpose and required scope, but lacks details on parameter interactions (e.g., handle restrictions) and return behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers all parameters with descriptions (100% coverage), so the baseline is 3. The description lists the parameters but adds no additional meaning beyond their names. The access scope is a tool-level detail, not parameter-specific.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update'), the resource ('existing navigation menu'), and the specific fields that can be modified ('title, handle, and items'). It also specifies the required access scope, which distinguishes it from other menu-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use the tool (to update an existing menu), but does not explicitly exclude alternatives or mention prerequisites. The schema hints at handle restrictions for default menus, but this is not in the description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behaviors. It does state 'Only provided fields are changed,' indicating a partial update (PATCH) behavior. However, it does not disclose required permissions, error conditions, or other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys purpose and a key behavioral trait (only provided fields change). No wasted words, though it could include a brief usage note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter update tool with no output schema, the description is minimal. It covers the essential partial update behavior but lacks information about return values, prerequisites, or potential errors. Schema fills param meanings but not all context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for each parameter. The description adds the concept of partial update but no further per-parameter details. Baseline 3 is appropriate as schema already provides meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (update) and the resource (existing product variant). It distinguishes itself from siblings like create_variant and delete_variant by specifying it updates an existing variant. The phrase 'Only provided fields are changed' adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for updating variant fields but does not provide explicit when-to-use or when-not-to-use guidance. No alternatives are mentioned among the many sibling tools. Adequate but lacks explicit context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose potential side effects (e.g., duplicate email handling, authentication needs, or rate limits). Beyond the creation action, minimal behavioral context is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. Front-loaded with the core action, then optional details. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic creation action and key parameters, but given 7 parameters (including a nested addresses object) and no output schema, it omits important details like the return value (created customer object) and uniqueness constraint on email. It is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description. The description adds value by clarifying which fields are essential (email is effectively required) and summarizing optional fields (name, phone, address, tags). This reduces ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new customer' (specific verb+resource) and distinguishes it from update_customer or other creation tools. The minimal requirement of email is explicitly mentioned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use (creating a customer) but does not provide explicit when-not-to-use or alternatives like update_customer. It is adequate but lacks exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses a constraint ('Must specify at least one option value') but lacks detail on side effects (e.g., inventory adjustments), auth requirements, error conditions, or idempotency. The write operation is implied but not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences with zero waste. First sentence states purpose, second adds critical constraint. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 13 parameters and no output schema, the description is adequate but incomplete. It doesn't mention what the response contains (the created variant object), nor does it cover behavioral or error scenarios. Some contextual gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by noting the constraint that at least one option value must be specified, which goes beyond the schema's required field list (only product_id marked required). This helps agents understand parameter dependencies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new variant for a product,' specifying the verb (create) and resource (variant). It distinguishes from sibling tools like create_product and create_bundle by targeting a specific entity. The added constraint 'Must specify at least one option value' further clarifies scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (adding a variant to a product) but provides no explicit guidance on when not to use or comparison with alternatives like update_variant or create_product. No prerequisites or context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states a read operation but does not disclose error behavior, return value structure, or permissions required. The example GID is helpful but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource, includes an example. No unnecessary words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and no output schema, the description covers the identifier format but does not describe the return value (expected to be a discount object) or error cases. Some context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. The description adds value by providing a concrete GID format example ('gid://shopify/DiscountAutomaticNode/123'), which is more specific than the schema's description 'GID of the automatic discount node.'
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb 'Get' and the resource 'automatic discount' with a concrete example of the GID format. It distinguishes this tool from sibling tools like get_code_discount and list_automatic_discounts by focusing on single retrieval by GID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., list_automatic_discounts, get_code_discount). The agent must infer usage from the name and sibling list, which is insufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states retrieval of info without disclosing side effects, auth needs, or response structure. Minimal transparency beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action, lists key fields. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output schema and doesn't specify return format or error cases. For a simple read tool, it's minimally viable but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, schema coverage is 100%. Description adds value by listing returned fields but doesn't need to explain parameters. Baseline score of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states retrieval of store info with specific fields listed (name, domain, email, etc.), distinguishing it from sibling tools like get_product or get_customer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance, but it's implicitly clear as a store-level info retrieval tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It implies a read operation by stating 'Get full details' but does not explicitly confirm it is non-destructive or require authentication. The phrase 'including its rules' adds some transparency about the response content.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that conveys the purpose without any unnecessary words or repetition. It is appropriately front-loaded and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description gives a reasonable idea of what is returned ('full details... including its rules'). However, it could be more thorough by mentioning the return format or confirming it returns a smart collection object. The siblings list provides context but the description itself is slightly thin.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter 'collection_id' is fully described in the schema with a clear explanation. The description adds no additional meaning beyond what the schema provides, which is already complete. Schema description coverage is 100%, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'smart collection', and the scope 'by its ID, including its rules'. It distinguishes itself from sibling get tools like 'get_custom_collection' by specifying 'smart collection'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates you need the ID to fetch a single smart collection, but provides no guidance on when to use this versus listing tools (e.g., list_smart_collections) or other get tools. No exclusions or alternatives are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states it returns available quantities but does not mention pagination behavior, rate limits, or whether it lists all items or only those with stock.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences (24 words), concise and front-loaded with the core purpose. No filler or unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description vaguely indicates the return format (available quantities per item) but lacks specifics on the structure (e.g., fields like item ID, quantity). Pagination details are also missing, though implied by the page_info parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The tool description does not add additional meaning beyond the schema descriptions for location_id, limit, and page_info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists inventory levels (stock quantities) for a given location and specifies that it returns available quantities per item. It is distinct from sibling tools like list_locations or list_products.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for checking stock quantities at a location but provides no explicit guidance on when to use this tool versus alternatives like adjust_inventory or set_inventory.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions the GraphQL interface and that it returns specific fields, but does not disclose pagination behavior, default limit, or read-only nature. The description says 'List all metafields' which may be misleading given the limit parameter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the action and key details (GraphQL, returned fields, optional filter). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description partially covers return fields but omits pagination, limit defaults, and result count. The list operation's behavior is not fully described considering the 3 parameters and typical expectations for list endpoints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds minimal value beyond restating the namespace filter option and product ID type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all metafields attached to a product via GraphQL.' It specifies the resource (metafields) and verb (list), and distinguishes from siblings like get_product_metafield and list_metafield_definitions by focusing on product metafields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for listing product metafields but does not explicitly state when to use this vs alternatives (e.g., get_product_metafield for a single metafield) or provide exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the tool uses metafieldsSet and performs an upsert, implying it can overwrite existing metafields. However, it does not disclose potential side effects, authorization needs, or behavior when the metafield exists vs. doesn't exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous words, clearly front-loading the purpose. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 5-parameter upsert tool with no output schema or annotations, the description is brief. Lacks details on return values, namespaces, or error conditions. Adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the parameters are well-documented in the input schema. The description adds minimal meaning beyond 'upsert' and does not elaborate on parameter roles (e.g., type required for creation). Baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('create or update') and resource ('customer metafield'), and specifies the underlying API method (metafieldsSet, an upsert). This distinguishes it from sibling tools like get, delete, or list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions it's an upsert (create or update) but provides no explicit guidance on when to use this tool vs. alternatives like create_metafield_definition or update tools. No prerequisites or context for selection given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It discloses the partial update characteristic but does not mention required permissions, error behavior, rate limits, or other operational traits. Minimal but adequate for a simple update.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short—one sentence—with no waste. It is front-loaded with the primary verb. Could be slightly more structured, but efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update tool with 7 parameters, no output schema, and no nested objects, the description covers the key behavioral trait (partial update). It is reasonably complete, though it omits potential side effects or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the schema's parameter descriptions. The phrase 'Only provided fields are changed' is already implied by the schema's optional fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing customer and specifies the partial update behavior. It is a specific verb+resource and distinguishes from create or delete customer tools in the sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives like create_customer or update_customer_metafield. It implies usage for updating customers, but no when-not or context for alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits beyond the action itself. It lacks details about side effects, permissions, idempotency, or what happens if the discount is already active.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no redundant information. Every word is essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description is adequate. However, it lacks information about idempotency and error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents the 'id' parameter. The tool description adds no additional meaning beyond what is in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Activate' and resource 'automatic discount', and clarifies the scope (currently inactive or scheduled), which distinguishes it from siblings like deactivate_automatic_discount or create_automatic_discount_basic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool (for inactive or scheduled automatic discounts), but does not provide explicit exclusions or alternatives like deactivate_automatic_discount.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the creation behavior and rule inclusion but omits details like idempotency, permissions, or side effects. It is minimally transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-formed sentence that communicates the core purpose efficiently with no unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich parameter schema and no output schema, the description is mostly complete. It could mention the return value (e.g., created collection ID), but it still provides enough context for an agent to understand the tool's function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds only marginal value by contextualizing the rules parameter. It does not significantly enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create), the resource (smart collection), and its distinguishing feature (automatic rules). It effectively differentiates from custom collections and other creation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lacks explicit guidance on when to use this tool versus alternatives like create_custom_collection. It implies the use case (automatic rules) but does not provide when-not-to-use or context about prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description omits behavioral details such as side effects (e.g., starting data transmission), rate limits, or validation of the address URL. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences with no fluff, but could benefit from a slightly more structured format (e.g., bullet points for examples). Still concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Missing output schema and description of return value (likely a webhook object). Also lacks notes on address validation or idempotency. Adequate but not complete for a creation tool with no annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by listing common topics, providing context beyond the schema. The schema already defines topic, address, and format adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a new webhook subscription' with specific verb and resource, and provides common topic examples that differentiate it from sibling tools like delete_webhook or list_webhooks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides common webhook topics, aiding the agent in deciding when to use this tool. However, it does not explicitly state when not to use it or mention prerequisites like API permissions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the non-destructive behavior ('without deleting it') and hints at reversible pausing, but does not mention side effects, authorization needs, or impact on ongoing discounts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words, efficiently conveying the core action and key distinction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description covers the essential action and non-destructive nature, though it could optionally mention prerequisites like the discount must be active.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'id,' which is described as 'GID of the automatic discount node.' The description adds no additional meaning beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Deactivate (pause) an automatic discount without deleting it,' which is specific and distinguishes it from related siblings like 'activate_automatic_discount' and 'delete_automatic_discount.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for temporarily disabling a discount without deletion, but it lacks explicit guidance on when to use it versus alternatives or any conditions or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations present, so description bears full responsibility. Warns that deletion is permanent and cannot be undone, which is critical. Lacks other traits (e.g., required permissions, effects on active discounts, error handling).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no redundant information. Action and critical warning front-loaded. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter delete operation with no output schema, the description is nearly complete. Missing details about non-existent entities or side effects, but the core purpose and irreversibility are clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (id parameter documented as 'GID of the automatic discount node to delete'). Description adds no additional meaning beyond restating the purpose, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (permanently delete) and the resource (automatic discount) with identifier (GID). Distinct from sibling tools like activate/deactivate or delete_code_discount.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. The irreversibility warning is helpful but doesn't differentiate from other delete tools or specify prerequisites. Context from sibling names implies use case but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description adequately discloses key behavioral traits: the action is irreversible ('cannot be undone') and requires a specific access scope ('write_files'). This is sufficient for an agent to understand the destructive nature, though it could mention what happens to related data or error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, containing two short sentences with no extraneous words. It front-loads the core action and immediately follows with critical caveats, making it efficient for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, no output schema), the description covers the essential operational context: what it does, the irreversible effect, and access requirement. It lacks details on return values or error conditions, but those are not critical for this straightforward deletion action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not add additional meaning beyond the input schema, which already has a thorough parameter description for 'file_ids' including the format example and minItems constraint. Since schema coverage is 100%, a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Permanently delete'), the resource ('files'), the method ('by their GIDs'), and includes key caveats ('cannot be undone', 'requires write_files access scope'). It distinguishes itself from other file-related tools like create_file or update_file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when to use delete_files vs. other deletion tools or whether there are safer alternatives. It only states the permanent nature but does not help the agent decide when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It discloses the destructive action and the constraint that a product must have at least one variant. However, it omits details like reversibility, permissions, or error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the action and include an important constraint. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with two required params and no output schema, the description covers the essential constraint. It lacks return value or error handling info, but that is not critical for a delete action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (both parameters explained in schema). The description adds no further parameter-specific meaning, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes a product variant and adds a critical constraint about minimum variant count. This distinguishes it from create/update/get variants.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (deletion) but does not explicitly advise against deleting the last variant or suggest alternative tools like create_variant or update_variant. The constraint hints at a condition but lacks clear guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions using GraphQL but lacks disclosure of behavioral traits such as authorization needs, rate limits, or side effects. As a read operation, it should have stated its non-destructive nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, and contains no extraneous information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description covers the core functionality and return content (components, quantities, option mappings). It could mention error handling or prerequisites, but is generally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides a full description of the product_id parameter (100% coverage). The tool description adds no extra meaning beyond what is in the schema, meeting the baseline but not exceeding it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves full details of a bundle product, including components and quantities. It distinguishes itself from siblings like get_product by specifying it's for bundles.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies it's for bundle products by mentioning 'bundleComponents', but does not explicitly state when to use this tool versus alternatives like get_bundle_operation or get_product. No when-not guidance is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations; description is minimal. Discloses it's a read operation but doesn't mention behavior on missing metafield (e.g., returns null vs error) or any preconditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, zero wasted words. Efficient and front-loaded with core information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple; description is mostly adequate. Lacks mention of return format or behavior when metafield not found, but schema covers parameters well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 3 parameters with descriptions. Description adds no additional meaning beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'get', resource 'single order metafield', and method 'by namespace and key'. It distinguishes from sibling tools like list_order_metafields (list multiple) and set/delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use guidance. Implies use when namespace and key are known, but does not contrast with list_order_metafields or other options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It states the return details but does not disclose permissions, rate limits, or error conditions (e.g., what happens if product ID does not exist). It adequately conveys it's a read operation returning comprehensive data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no extraneous content. It is front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, the description covers the key output elements (variants, images, options). It lacks details on response format or errors, but these are arguably covered by the schema and typical REST conventions. Adequate for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter with a description, and the tool description adds 'by its ID' but not additional semantics. With 100% schema coverage, baseline is 3; the tool description does not add significant value beyond the schema for parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get' and the resource 'product by its ID', and specifies the scope includes 'all variants, images, and options'. This distinguishes it from siblings like get_variant or get_product_image.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies you need the product ID to use this tool, but it does not provide explicit guidance on when to use this tool versus alternatives like list_products or get_variant. No exclusions or conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. Mentions access scope requirement but lacks details on error cases, idempotency, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 12 words, conveys purpose and prerequisite. No wasted information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple get operation with one param and no output schema. Could mention return format or error behavior, but not strictly necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter with clear schema description. Description adds no extra meaning beyond schema, which covers 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states action (get), resource (details of a single theme), and identifier (GID). Distinguishes from sibling tools like list_themes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions required access scope but does not explicitly say when to use vs alternatives like list_themes or get_theme_files. Implicit guidance via context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses read-only nature via access scope and 'Get' verb, and mentions full content inclusion. Lacks details on error handling, rate limits, or behavior when file not found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No unnecessary words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet description does not specify return format (e.g., object with filename and content). Missing error scenarios, pagination if many files, or confirmation of success. For a data retrieval tool, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters (theme_id format, filenames array with examples). Description adds no additional parameter-level meaning beyond noting that full content is returned.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Get', resource 'theme files', and action 'by filenames, including full content'. Distinguishes from sibling list_theme_files and upsert_theme_files. Requires read_themes access scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions required access scope (read_themes). Does not explicitly state when not to use or provide alternatives, but context from sibling names makes it clear that list_theme_files is for listing and upsert for writing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description lacks disclosure of behavioral traits such as read-only nature, authentication requirements, or rate limits. It only states that it lists locations, which is obvious from the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, front-loading the purpose without any unnecessary words. Every sentence provides value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains what the tool lists and its relevance to inventory operations. For a simple list with no parameters and no output schema, it is adequate but could mention whether there are any filters or limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema coverage is 100%. Per guidelines, a baseline of 4 is appropriate since no parameters need documentation, and the description adds no extra parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that it lists all locations for the Shopify store and specifies that each location has an ID used for inventory operations, aligning with the tool's name and distinguishing it from other list_ tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for obtaining location IDs for inventory operations but does not explicitly mention when to use this tool versus alternatives like get_location or adjust_inventory.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It indicates a read-only listing operation (implied by 'List') and mentions 'via GraphQL', but does not disclose pagination, rate limits, permissions, or any side effects. The description adds some context (store-level scope) but falls short of full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences: first states purpose, second lists return fields. It is front-loaded with the primary action and contains no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and no output schema, the description adequately states the purpose and return fields. It does not mention pagination or default behavior, but given the simplicity, it is nearly complete. Slightly lacking for full contextual completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with a single optional parameter 'namespace' well-described in the schema. The description adds value by stating the return fields, but does not elaborate on the parameter beyond what the schema provides. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists metafield definitions for products at the store level, specifying the resource and scope. It differentiates from sibling tools like list_product_metafields (which list values per product) by focusing on definitions. The return fields are listed, adding specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for viewing available metafield definitions, but it does not explicitly state when to use it versus alternatives like list_product_metafields or list_customer_metafields. No exclusions or context about when not to use it are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It states the tool lists pages and supports filtering, implying a read operation, but does not explicitly disclose read-only behavior, pagination, or potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, front-loading the purpose and adding necessary details without wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers main capabilities and required scopes, but lacks mention of pagination behavior or return format, which would be helpful given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds a summary of filter capabilities, but it largely repeats what the schema already provides without adding significant new meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists online store pages with specific examples, distinguishing it from sibling tools like get_page and other list tools for different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions required access scopes but does not provide explicit guidance on when to use this tool versus alternatives like get_page for single pages or other list operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds minimal behavioral context beyond annotations (none provided). It discloses the access requirement but lacks details on side effects, error handling, or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with one sentence and a scope requirement, containing no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (update name only) and no output schema, the description is fairly complete, though it omits return value information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds no extra meaning beyond what the input schema already provides for the two parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and the specific resource ('a theme's name'), distinguishing it from sibling tools like publish_theme or delete_theme.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It mentions the required access scope (write_themes) as a precondition, but does not provide guidance on when to use this tool versus alternatives, nor when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states that variants have price, inventory, and options, adding behavioral context. However, it does not disclose pagination behavior or the effect of the 'limit' parameter beyond schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the purpose, second explains what a variant is. No waste, front-loaded, and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a basic list tool with schema-covered parameters. Missing details about return format and pagination behavior, but sufficient given sibling tool clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for all parameters. The description adds minimal beyond the schema, mainly reinforcing variant attributes. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Describes the tool's purpose as listing variants of a product, specifying that variants are purchasable SKUs with price, inventory, and options. This clearly distinguishes it from sibling tools like 'list_products' or 'get_variant'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context that this tool is for retrieving all variants of a product. However, it does not explicitly mention when not to use it or alternatives like 'get_variant' for a single variant.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses mutation nature and state/scope requirements but does not detail return values or side effects like whether updates are irreversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. Front-loaded with key actions and prerequisites. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description covers purpose and prerequisites but lacks information on return values or success indicators. Ambiguity about updating multiple fields simultaneously (description says 'or' but schema allows multiple).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all parameters have descriptions). Description adds no new parameter details beyond summarizing what fields can be updated, so baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states updates can be made to alt text, filename, or content via a new source URL. It specifies the resource (file) and actions, distinguishing it from create_file and delete_files among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit prerequisites: file must be in READY state and requires write_files access scope. Does not explicitly mention when not to use or list alternatives, but guidelines are clear for a simple update tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description adds the critical partial update detail but omits other important behaviors such as required permissions, side effects (e.g., webhooks), or error handling for invalid fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words, directly conveys the core functionality. Front-loaded with the action and key behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of updating a product with 7 parameters and no output schema, the description adequately covers partial update but lacks mention of return value or validation outcomes. Incomplete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description's partial update note adds marginal context beyond the schema's field descriptions, but not enough to raise the score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing product and emphasizes partial update behavior, distinguishing it from related tools like create_product or delete_product.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions that only provided fields are changed, giving a clear usage pattern. However, it lacks direct comparison with alternative tools or when-not-to-use scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool creates discounts with targeting, minimum requirements, and combination rules, but does not discuss side effects, permissions, or limits like daily usage caps or discount code uniqueness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences concisely capture the tool's purpose and core features with no redundancy. Information is front-loaded and each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 16 parameters and no output schema, the description provides a sufficient high-level overview. It covers the main capabilities but does not explain return values or error conditions, which is acceptable for a creation tool with rich schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no extra detail beyond summarizing the overall purpose; it doesn't clarify edge cases or provide examples for complex fields like 'combinesWith'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it creates an amount-off code discount (percentage or fixed), distinguishes from sibling tools like bxgy and free shipping, and specifies key features like targeting and combination rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for amount-off discounts, differentiating it from bxgy and free shipping siblings, but does not explicitly state when to use this tool vs alternatives or provide any exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly indicates the action (create) and the resource (page). It also discloses required permissions. However, it does not mention potential side effects, idempotency, or what happens on duplicate titles, but these are not critical for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences and no extra words. It efficiently conveys the purpose and a key requirement. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 6 parameters, no output schema, and no annotations, the description covers the essential purpose and required scopes. It does not discuss return values or default behavior (e.g., auto-generated handle), but the schema covers those. Overall, it is complete enough for an agent to understand the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, meaning the schema already documents all parameters thoroughly. The description adds no extra meaning beyond what's in the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new online store page' with examples like About Us, Contact, policy pages. This is a specific verb+resource combination, and the resource is unique among siblings (no other page creation tool exists).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the required access scopes (write_content or write_online_store_pages) which is helpful, but it does not provide guidance on when to use this tool versus alternatives or any context for when not to use it. Without sibling differentiation, this is adequate but not strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly states the action is permanent and irreversible ('This cannot be undone'), providing critical behavioral context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence clearly communicates purpose and key behavioral trait, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple delete tool; missing output info but not essential. Could mention required permissions, but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers parameter fully (100% coverage), so description adds no extra detail beyond the schema's GID format example.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Permanently delete') and the resource ('a code discount by its GID'), making it distinct from sibling tools that delete other entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like deactivate_code_discount, but the context implies permanent removal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It clearly indicates the operation is permanent and that products are unaffected, providing essential behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with front-loaded action. Every word is useful with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter, the description adequately covers the effect and what is not affected. No output schema or annotations needed further elaboration.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%. The description adds no additional meaning beyond what the schema already provides for the collection_id parameter. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'permanently delete' and specifies it is for custom collections. It also explicitly notes that products are not deleted, distinguishing it from related tools like delete_product.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage (deleting a collection without affecting products), but does not explicitly state when to use this tool versus alternatives like remove_product_from_collection or delete_product.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the action is permanent and irreversible, which is critical for a destructive operation. However, it could mention side effects on associated data like variants or images.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using only two sentences to convey the purpose and irreversibility. It is front-loaded with the most critical information, and every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has only one parameter and no output schema or annotations, the description covers the essential aspects: what it does and its permanent effect. Some additional context about related data (e.g., product images, variants) would improve completeness, but the current level is adequate for a simple delete operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 100% with a single parameter (product_id) that is already well-described in the schema as 'The numeric Shopify product ID to delete.' The description adds no additional meaning beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Permanently delete a product by its ID', specifying the verb (delete), resource (product), and a key characteristic (permanent). This distinctly separates it from sibling delete tools that target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that the tool should be used to permanently delete a product, but it does not explicitly discuss when to use it versus alternatives, nor does it mention prerequisites or conditions under which deletion should be avoided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the burden. It discloses the key constraint and permission requirements, but does not describe any side effects or return behavior beyond the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states the action, second adds essential constraints. No fluff, front-loaded, efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple single-parameter input and no output schema, the description sufficiently covers the key constraints and permissions. It is adequate for an agent to use correctly, though it does not detail the result of a successful deletion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (theme_id) with 100% schema coverage; the description adds no additional meaning beyond the schema's own description, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete a theme' which is a specific verb+resource. The additional constraint about the active (MAIN) theme not being deletable distinguishes it from other delete tools among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the active theme cannot be deleted and the required permissions (write_themes scope and Shopify exemption), providing clear context for use. No direct comparison to alternatives, but the conditions are well-defined.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It states it 'gets' a discount, implying a read-only operation, but lacks details on permissions, side effects, or return specifics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loading the action and identification options with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter lookup tool with no output schema, the description covers the essential purpose and identification methods. It does not describe return values, but that is acceptable given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameters have descriptions. The tool description adds minimal additional meaning beyond the schema, just clarifying that 'code' is customer-facing, which the schema already indicates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get a code discount' and specifies two identification methods (GID or code string), distinguishing it from siblings like list_code_discounts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (when you have a GID or code string), but does not explicitly mention when not to use it or provide alternative tools for listing or modifying discounts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description indicates a read operation ('get full details') but provides no information about permissions, error handling, or side effects. With no annotations, the description does not fully disclose behavioral traits beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 10 words, front-loading the verb and resource. No unnecessary information, achieving maximum conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description is adequate. It explains what the tool does and its input, though it could mention the return format or common errors.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one parameter fully described). The description adds no semantic value beyond the schema, simply restating 'by its ID'. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('get') and resource ('custom collection'), and specifies it's a single collection by ID, distinguishing it from sibling tools like list_custom_collections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing full details of a specific custom collection by ID. It does not explicitly exclude alternatives, but the sibling list provides context (e.g., get_smart_collection, list_custom_collections).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It discloses pagination behavior ('cursor-based') and limit, but does not detail the return format, default sorting, or whether it requires authentication. It is a read operation, so no destructive effects are needed, but more context on permissions would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the primary action, and includes essential details (filters, pagination) without any extraneous words. Every sentence is justified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and full schema coverage, the description is complete. It mentions pagination and filterable fields. It lacks guidance on default sorting or response structure, but for a list tool with no output schema, it is sufficiently comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. The description adds minimal value: it restates the `limit` parameter's effect and `page_info` for pagination. Since the schema already explains parameters well, the description's contribution is marginal, meeting the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List products'), the resource ('in the store'), and distinguishes from sibling list tools (e.g., list_orders, list_customers) by being specific to products. It also mentions optional filtering and pagination, which further clarifies scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells when to use the tool (listing products with optional filters), and the context of sibling tools implies it's for product-specific queries. It does not explicitly state when not to use it or mention alternatives, but for a straightforward list function, this is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It mentions required scope and file categories, but does not disclose pagination behavior beyond the cursor parameter, error handling, or rate limits. The input schema covers pagination, so transparency is adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the core purpose. Every word adds value with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with three parameters and no output schema, the description is mostly complete. It covers the resource, scope, and file types. It does not describe the return format or error scenarios, but these are often implied in list tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all three parameters. The description adds no additional parameter-level details beyond the schema. Baseline of 3 is appropriate as the schema already documents parameters well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'files in a theme', and lists specific file types (templates, assets, sections, snippets, config, locales). It also distinguishes itself from siblings like get_theme_files (specific file) and upsert_theme_files (create/update).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly requires 'read_themes access scope', giving clear context for when to use the tool. It does not explicitly state when not to use or list alternatives, but the scope requirement and file types provide sufficient guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It implies mutation by reordering but does not disclose behavioral traits like idempotency, error handling, authentication needs, or what happens to invalid product IDs. The precondition about sort_order is helpful but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the action and then providing instructions and a condition. Every sentence adds value with no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with 2 parameters, no output schema, and no nested objects, the description covers the purpose, instructions, and a critical precondition. It lacks details about the return value or expected result, but overall it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters described in the schema. The description adds the precondition about sort_order but largely reiterates the schema's description of the parameters ('Provide an ordered array of product IDs'). The additional value is marginal, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reorder products in a custom collection') and the resource ('products in a custom collection'). It distinguishes from sibling tools like add_product_to_collection, remove_product_from_collection, and list_collection_products by specifying reordering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a key precondition: 'The collection's sort_order must be 'manual' for positions to be honoured on the storefront.' This provides clear guidance on when the tool is effective, though it does not explicitly mention when not to use or alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It fails to mention that the tool is read-only, does not describe pagination behavior, rate limits, or any side effects. This is a significant gap for a search operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose. The first sentence states verb and resource, the second adds concrete examples. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (2 params, no nested objects, no output schema), the description is mostly complete. It covers the query format but does not mention that results are paginated or that the limit parameter exists (though schema covers it). Still adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, both parameters are described in the schema. The description adds value by specifying the Shopify search syntax with examples (e.g., 'email:john@example.com'), which goes beyond the schema's descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Search', the resource 'customers', and the mechanism 'using a query string'. It provides concrete examples of Shopify search syntax, distinguishing it from siblings like list_customers and get_customer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for flexible customer searching via query syntax but does not explicitly state when to use this tool versus siblings like list_customers (for listing all) or get_customer (by ID). No exclusion criteria are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description reveals the key behavioral trait of overwriting the current available quantity. It does not disclose potential side effects, required permissions, or error handling, but for a simple set operation this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is direct and free of superfluous information. It front-loads the key action and result, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a straightforward set operation with no nested objects or output schema, the description adequately covers the core functionality. It could mention error conditions or confirmation of the operation, but the current text is sufficient for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are fully described in the input schema, achieving 100% coverage. The description adds no additional parameter information beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets inventory to an absolute quantity at a given location, using the verb 'Set' and resource 'inventory'. It distinguishes itself from the sibling tool 'adjust_inventory' by specifying it overwrites the current available quantity, implying a delta vs absolute operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes that this tool overwrites the current quantity, indicating it should be used when a precise absolute quantity is needed rather than an incremental adjustment. However, it does not explicitly contrast with the sibling 'adjust_inventory' tool, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It mentions upsert behavior and a batch limit, but does not clarify side effects like whether existing files are overwritten or merged, nor does it describe the response. This is adequate but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and key constraints. Every sentence provides essential information: action, limit, content format, and access scope. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (2 well-documented parameters, no output schema, no annotations), the description is fairly complete. It covers what, how many, how, and who. A minor gap is the absence of response details, but overall sufficient for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description reiterates content formats (text/base64) already covered in schema, but adds the batch limit, which is also present in schema (maxItems: 50). No additional semantic meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create or update theme files' with a specific resource and action. The limit of 50 files per request distinguishes it from siblings like get_theme_files and delete_theme_files, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit instructions on how to provide content (text or base64) and required access scope. However, it does not offer guidance on when to use this tool versus alternatives like update_theme, and could clarify that it is the primary method for theme file modifications.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It explicitly states the irreversible destructive nature ('cannot be undone'), which is critical. However, it omits potential error conditions, authorization needs, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with only two sentences that convey all essential information without redundancy or extraneous details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool simplicity (one required parameter, no output schema, no nested objects), the description is fully adequate. It covers purpose, action, irreversibility, and parameter identification.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for the single parameter, so the baseline is 3. The description does not add additional meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Permanently delete'), the resource ('metafield'), and emphasizes irreversibility ('cannot be undone'). It effectively distinguishes from sibling tools like set or get metafield operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While the description implies when to use (when permanent deletion is needed), it does not explicitly contrast with alternatives like updating or deactivating metafields. No guidance on prerequisites or confirmation is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
As a read-only tool, the description implies safety but does not explicitly state no side effects, idempotency, or authentication requirements. No annotations are provided, so the description carries full burden; it is adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence with no filler. Verb and resource are front-loaded, and every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, the description sufficiently hints at return structure by listing example fields, making it complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage with a clear description of order_id. The tool description adds no extra meaning beyond restating 'by its ID', so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the action (Get), the resource (single order by ID), and lists key included details (line items, shipping address, etc.), clearly distinguishing it from sibling tools like list_orders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implicitly clear from context: use when needing full details of a single order. Sibling list_orders handles multiple orders, but no explicit when-not or alternative guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility. It explains the tool's role in the upload flow and the need for write_files scope. However, it does not disclose details about the pre-signed URL's lifetime, size limits, or potential side effects of repeated calls, leaving some behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, covering purpose, usage steps, and prerequisites in a single well-structured paragraph. It front-loads the core purpose and uses numbered steps for clarity, with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description only vaguely references the response (upload URL, parameters, resourceUrl). It does not detail the exact fields returned, which may leave the agent uncertain about what to expect. However, the step-by-step guide partially compensates by specifying how to use the output in subsequent steps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining the resource enum options (e.g., 'Use IMAGE or FILE for general assets') and noting that file_size is required for VIDEO and MODEL_3D. This provides context beyond the basic schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as generating a pre-signed staged upload target and explicitly positions it as step 1 of Shopify's two-step upload flow. It distinguishes itself from sibling tools like create_file by describing the overall workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear step-by-step instructions for when to use this tool (as the first step) and references the next step (create_file). It also mentions required access scope (write_files). However, it does not explicitly state when not to use this tool or mention alternatives beyond the implied flow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It states that only provided fields are changed, which is a key behavioral trait. However, it does not mention idempotency, permissions, error conditions, or return value. This is adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. It front-loads the action and key behavior without any unnecessary words. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (5 parameters, no output schema, no annotations), the description is somewhat complete. It covers the core behavior but omits what the tool returns, prerequisites (e.g., draft order must exist), and error scenarios. With no output schema, more detail would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so baseline is 3. The description adds value by stating the partial update semantics ('Only provided fields are changed'), clarifying that optional parameters are merged rather than replaced. This goes beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'draft order', and specifies that only provided fields are changed. This distinguishes it from sibling tools like create_draft_order, delete_draft_order, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it (to modify an existing draft order) and the partial update behavior. It does not explicitly mention alternatives, but the sibling tool names provide enough context. A more explicit comparison could improve it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description explains mutation effect but does not disclose edge cases (e.g., negative inventory handling), permissions, or reversibility. Basic transparency but could be more comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no extraneous information. Front-loaded with action and key constraints. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple mutation tool with 3 parameters and no output schema. Explains purpose, parameters, and prerequisites. Could mention expected return but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds value by summarizing adjustment direction ('positive to add') and source of inventory_item_id. Enhances understanding beyond raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it adjusts inventory by a relative amount, with explicit positive/negative semantics. It identifies required IDs and distinguishes from siblings like set_inventory (absolute) and list_inventory_levels (read).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: requires inventory_item_id and location_id. Implies when to use (adjusting stock relatively) but does not explicitly exclude alternatives like set_inventory for absolute adjustments.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the asynchronous nature and necessary polling, compensating for the lack of annotations. However, it does not detail potential failures or side effects like duplicate detection, which would enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at three sentences, front-loaded with the primary purpose, followed by critical async behavior and requirements. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential workflow: creation, async handling, and prerequisites. Without an output schema, it could mention what the response contains (e.g., operation ID), but the polling instruction implies it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the description adds minimal value beyond the schema. It does not explain any parameter semantics not already covered in the schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create a fixed product bundle' and differentiates from siblings like update_bundle and get_bundle_operation by specifying it creates a new bundle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides prerequisites (Shopify Bundles app, write_products scope) and post-usage guidance (poll get_bundle_operation), but does not explicitly list when to avoid using this tool or compare to other create tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It states that new themes are UNPUBLISHED by default and requires 'write_themes' scope. However, it does not mention return value, side effects, or whether the operation is synchronous, but the given details are sufficient for a basic create operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The first sentence states the core action and source types; the second adds default behavior and required scope. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (3 parameters, no output schema), the description covers the key aspects: what it does, how to supply the source, the default role, and access scope. It does not describe the return value or errors, but this is acceptable for a straightforward creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds context beyond the schema: it clarifies that 'source' can be a public ZIP URL or staged upload URL, and explains the 'role' enum values (default UNPUBLISHED, DEVELOPMENT for temporary use). This additional semantics justifies a score of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'create', the resource 'theme', and the mechanism ('from a public ZIP URL or a staged upload URL'). It distinguishes this tool from sibling tools like update_theme, publish_theme, and delete_theme by specifying the creation action and default status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly guide when to use this tool versus alternatives such as stage_upload or create_theme_files. It implies the need for a URL but lacks 'when not to use' or comparison with other create tools for different resources.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description provides important behavioral context: required access scope and a constraint (default menus cannot be deleted). It does not describe side effects or reversibility, but for a simple deletion this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the action, and each sentence provides essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a deletion tool without output schema, the description covers the action, constraints, and auth requirement. It could mention the permanence of deletion, but overall it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage describing the 'id' parameter. The description adds no additional meaning beyond what the schema already provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes a navigation menu, specifies that default menus cannot be deleted, and mentions required access scope. This distinguishes it from sibling tools like create_menu or update_menu.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use (deleting non-default menus) and when not to use (default menus). It mentions the required access scope. However, it does not explicitly contrast with alternatives like updating a menu to hide it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses the destructive nature ('Permanently delete', 'cannot be undone') and required authorization scopes. This is adequate for a simple delete tool. No contradictions exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two sentences that deliver essential information without redundancy. The first sentence states the action and scope, the second adds irreversibility and permissions. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the key aspects: action, irreversibility, and required permissions. It could be improved by mentioning the expected return value or confirmation behavior, but it is sufficient for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no new information beyond what the schema already provides for 'page_id'. The schema description already states 'The numeric Shopify page ID or GID to delete.' The tool description repeats this concept without elaboration.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Permanently delete an online store page.' It distinguishes from sibling tools that delete other entities (e.g., delete_product, delete_theme) by targeting pages specifically. The verb 'delete' plus resource 'page' is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides important usage context: the operation is permanent ('cannot be undone') and requires specific access scopes ('write_content or write_online_store_pages'). This helps the agent decide when it's appropriate to use. However, it does not explicitly contrast with alternatives or state when not to use, which could be improved.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses that the tool returns details including access scopes and active subscriptions. As a GET operation, no side effects expected, but no mention of permissions or limits. Still sufficiently transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, 16 words, front-loaded with verb and resource. No extraneous information. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description mentions key return fields. For a simple get tool, this is adequate. Could include error handling or full field list, but sufficient for typical use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter description. The description only restates using GID, adding no new semantics. Baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves details of a specific app installation using its GID, listing included data (access scopes, active subscriptions). It distinguishes from sibling 'list_app_installations' which lists all installations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for retrieving specific installation details. No explicit when-not or alternatives, but context with sibling name suggests differentiation. Adequate guidance for a simple read tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses it returns 'full details' and 'body HTML', which is adequate for a read operation. It does not mention rate limits or other traits, but the simplicity mitigates the need.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, efficient and front-loaded: purpose and output in first sentence, required scope in second. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read tool without output schema or annotations, the description covers the core purpose, output (body HTML), and prerequisites. It is sufficiently complete for effective selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one parameter described), baseline 3. The tool description does not add extra meaning beyond the schema's parameter description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get full details'), the resource ('single online store page by its ID'), and a key output ('including body HTML'). It distinguishes from sibling tools like list_pages or update_page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes required access scopes ('requires read_content or read_online_store_pages'), providing a prerequisite. While it doesn't explicitly state when not to use it, the context is clear for a simple retrieval operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It discloses the upsert behavior and the condition for using `type`. However, it does not mention potential side effects, idempotency, error handling, or rate limits, which would be valuable for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with front-loaded action. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential behavior and parameter guidance. It could be more complete by addressing error scenarios or what happens if the product doesn't exist, but it is sufficient for a tool with a well-documented schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining when to use the `type` parameter and specifying the GraphQL mutation. This goes beyond the schema's descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates or updates a metafield using the GraphQL `metafieldsSet` mutation (upsert). It distinguishes itself from sibling tools like `delete_product_metafield` and `get_product_metafield` by specifying the exact action and resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: it explains the upsert behavior and when to use the `type` parameter. However, it does not explicitly mention when not to use this tool or compare it to alternatives like `set_customer_metafield`, though the tool name and sibling list make this somewhat clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and discloses partial update behavior and auth requirements. Lacks details on idempotency or return value but covers core traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no filler. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately covers partial update and auth for a standard update tool with well-documented schema, though missing return value info.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds no parameter-specific meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it updates an existing online store page and specifies partial update behavior. Distinguishes from sibling tools like create_page and delete_page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides guidance on partial update ('Only provided fields are changed') and required access scopes, but does not explicitly exclude conditions or compare to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description effectively discloses key behaviors: default draft status, that images/variants are added later, and that partial failures leave the product intact. It does not cover permissions or rate limits, but the provided context is substantial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences efficiently cover purpose, requirements, and workflow. No redundancy; front-loaded with essential info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description explains creation, default status, and error handling, it lacks mention of the return value on success (no output schema). For an 8-parameter tool without output schema, this missing info reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters. The description adds value by explaining the role of variants (prices, SKUs, inventory) and reinforcing the workflow, which helps agents understand usage beyond raw schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new product' with specific details on required fields and default status. It distinguishes from sibling tools like 'create_variant' and 'create_product_image' by noting they are follow-up calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use this tool and outlines the sequence for adding images and variants. Mentions error recovery but does not explicitly state when not to use or list alternatives beyond the implicit workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
As a read operation, the description implies no side effects. However, pagination behavior (via limit parameter) is not explicitly described, though the parameter schema provides details. No annotations are present to contradict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose, second adds filter guidance and scope. No extraneous words, front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description adequately covers purpose, filter usage, and access scope. It could mention the return format (array of theme objects), but the tool is simple and the omission is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so baseline is 3. The description adds value by explaining role filtering with examples ('MAIN for published theme, development themes'), enhancing comprehension beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all themes installed on the online store' and specifies filtering by role, distinguishing it from single-theme get_theme and other theme-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description notes the required 'read_themes access scope' and gives examples of role filtering, but does not explicitly mention when not to use it or suggest alternatives like get_theme for a single theme.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses a key behavioral constraint (topic immutability). However, it does not mention other typical aspects like idempotency, permission requirements, or side effects, though the simplicity of the operation partly compensates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, each serving a distinct purpose: stating the action and providing a critical constraint. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 2-parameter tool with no output schema, the description covers the essential purpose and a key limitation, making it complete for effective usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and already describes both parameters. The description adds minimal value by clarifying 'callback URL' and 'numeric webhook ID', but these are largely redundant with the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (update), resource (webhook subscription), and specific attribute (callback URL). It distinguishes the tool from siblings like create_webhook, delete_webhook, and get_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when not to use: topic changes cannot be made via this tool, and suggests an alternative (delete and recreate the webhook). This provides clear guidance for proper tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description covers key behaviors: it's a polling operation, lists possible statuses, and implies non-destructive behavior. Does not mention rate limits or error handling, but for a simple poll this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: three sentences, each with clear purpose. Front-loaded with action and resource, no redundant information. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple polling tool with one parameter and no output schema, the description is almost complete. It explains purpose, statuses, when to call, and prerequisites. Could optionally mention timeout or error conditions, but not essential.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (operation_id described in schema). The description adds value by explaining that the ID comes from create_bundle or update_bundle results, providing context beyond the schema's generic 'GID' description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it polls the status of an asynchronous bundle operation, using specific verb ('poll') and resource ('bundle create or update operation'). It distinguishes from siblings like get_bundle (which likely retrieves bundle details) by focusing on status polling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use: 'Call this after create_bundle or update_bundle until status is COMPLETE.' Also states required scope ('Requires read_products scope'). Provides clear context without mentioning alternatives, which is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden of behavioral disclosure. It reveals that the operation is asynchronous, that components fully replace existing ones, and that it requires the 'write_products' scope. This goes beyond the input schema by explaining the replace behavior and the need for polling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each serving a distinct purpose: stating the verb and resource, clarifying component behavior, and explaining async handling. There is no redundancy or unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description adequately explains the async nature and how to retrieve the result via get_bundle_operation. It also mentions required scope. While it does not detail return values, the polling instruction compensates. The description feels complete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining the behavior of the 'components' parameter (replaces all current components if provided, omitting leaves unchanged), which provides critical semantic context beyond the schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Update an existing product bundle' which uses a specific verb ('Update') and resource ('product bundle'), clearly distinguishing it from sibling tools like create_bundle or get_bundle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool (to update a bundle), how components work (replace all if provided, omit to leave unchanged), and the async nature with polling instructions. It could explicitly mention not to use for creation or reading, but the context of siblings and the description itself sufficiently imply usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![shopify-mcp MCP server](https://glama.ai/mcp/servers/untitled-developers/shopify-mcp/badges/card.svg)](https://glama.ai/mcp/servers/untitled-developers/shopify-mcp)

Score Badge

Copy to your README.md:

[![shopify-mcp MCP server](https://glama.ai/mcp/servers/untitled-developers/shopify-mcp/badges/score.svg)](https://glama.ai/mcp/servers/untitled-developers/shopify-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/untitled-developers/shopify-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server