Server Quality Checklist

Tool Scores

• page_createA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It does state the default unpublished/draft behavior, but this merely repeats the schema's isPublished default and field description. It says nothing about permissions, return values, side effects, or failure modes, which is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and every clause contributes meaningful information. There is no redundant filler or over-explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple create tool with four parameters and no output schema or annotations, the description covers purpose and default publication state. However, it omits what the tool returns (e.g., created page object or ID) and any post-creation behavior, leaving some context incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no extra meaning beyond the schema's parameter descriptions, merely referencing isPublished without elaborating on the other parameters or adding format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Create') and resource ('a new page in the Shopify store'), clearly distinguishing creation from sibling tools like page_update or page_list. It also immediately signals the key scope (new page) without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes it clear this is for creating a new page, providing context that it is the create counterpart to page_update. It does not explicitly name alternatives or exclusions, but the sibling list and 'new page' wording imply when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• page_updateA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation only says `destructiveHint: true`, which is generic. The description adds key behavioral transparency with 'Only provided fields change', clarifying partial update semantics. It does not describe side effects like broken links or unpublishing consequences, but this is a meaningful addition beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, each earning its place: purpose, partial-update behavior, and a critical prerequisite. There is zero fluff and information is front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an update tool with 5 parameters and no output schema, the description covers the essential context: what the tool does, how to obtain the required ID, and the key behavioral nuance. It does not explain return values, but none are defined, and the description is adequate for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all 5 parameters have descriptions in the schema). The description adds the useful global note that only provided fields change, which clarifies how omitting optional parameters behaves. This is a marginal addition beyond the schema, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Update an existing page') and specifies the exact fields that can be modified (title, body, handle, publish status). It distinguishes from the create sibling by emphasizing 'existing page', but does not explicitly name alternative tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a prerequisite ('Get the page GID from page_list'), which implies the required context for usage. However, it does not explicitly discuss when to choose this tool over alternatives like page_create or menu_update, so usage guidance is implied rather than fully stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• theme_update_fileA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation destructiveHint=true already discloses the destructive nature of the operation. The description's 'Overwrite' confirms this but adds no additional behavioral context (e.g., permissions, reversibility, failure modes). The theme_list prerequisite is a usage note, not a behavioral disclosure. With annotations covering safety profile, this is an appropriate score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, followed by a necessary prerequisite. Every word earns its place; there is no fluff or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple file overwrite tool, the description combined with annotations (destructiveHint) and full schema coverage provides sufficient context. The workflow dependency on theme_list is explicitly stated, and no output schema is needed for a write operation. The tool is adequately specified for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all three parameters, so the baseline is 3. The description adds minimal parameter semantics; the only hint is to get themeId from theme_list, which is already implied by the GID format in the schema. No extra meaning is added for filePath or fileContent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it overwrites one file in a Shopify theme, specifying the verb 'Overwrite' and the resource 'theme file'. It distinguishes itself from sibling tools like theme_read_file (read operation) and page_update (different resource type). The title and description are consistent and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a prerequisite: 'Get the theme ID from theme_list first.' This tells the agent when to use this tool relative to theme_list, establishing a clear workflow. While it doesn't explicitly name alternatives, the sibling context and the specific action make the usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• theme_read_fileA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already signals a safe read operation, and the description's 'read' is consistent. The description adds a small amount of context (single file, prerequisite) but does not disclose error behavior or permission requirements; given the annotation, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, one for purpose and one for prerequisite, with no redundant words. The key information is front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with a well-documented schema and a clear prerequisite. The description gives sufficient context for an agent to select and invoke the tool, though it does not explicitly describe the return format or error cases, which are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents both parameters with types and examples, and the description adds a cross-reference to theme_list for the themeId source. This extra guidance enhances parameter understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool reads the content of one file in a Shopify theme, using a specific verb ('read') and resource ('file in a Shopify theme'). It distinctively scopes to a single file and contrasts with the sibling theme_update_file by focusing on reading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It instructs the agent to obtain the theme ID from theme_list first, providing an explicit prerequisite. While it does not explicitly mention alternatives like theme_update_file, the prerequisite and read-focused language make the usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• menu_listA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations show readOnlyHint=true, which the description supports. It adds meaningful context about the returned handle-based identification and its role in updates, without any contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences that front-load the purpose and then provide usage context. Every phrase adds value, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless read-only list tool with no output schema, it sufficiently describes the return content (GIDs, handles, items) and downstream usage (menu_update). No significant information gap remains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters, so description does not need to explain any. Baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States exactly what it lists (the store's navigation menus) and specifies the returned fields (GIDs, handles, items). It distinguishes itself from sibling tools by explicitly referencing menu_update, making its role clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides strong contextual guidance by explaining that themes reference menus by handle and that menu_update relies on data from this tool. Lacks an explicit 'when not to use' but clearly implies use for retrieving menu data before editing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• menu_updateA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotation already marks destructiveHint=true, but the description adds essential context: 'Items not included are DELETED from the menu.' This explains the exact destructive behavior beyond the annotation and advises a safe workflow. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and the critical destructive caveat. Every phrase earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive update tool with full schema coverage and explicit behavioral warning, the description is complete. It tells the agent what to do, why, and how to avoid data loss. No output schema exists, but mutation tools don't require return-value documentation to be usable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers all parameters with descriptions (100% coverage), so baseline is 3. The description adds valuable semantic context by emphasizing that 'items' must be the FULL menu and that IDs must be included to preserve existing items, which supplements the schema's per-field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Replace') with a clear resource ('menu's items') and states the destructive replacement semantics. It clearly distinguishes menu_update from menu_list, which is about retrieving menus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides strong usage guidance: 'get the current items from menu_list first and include every item you want to keep, with their ids.' This establishes an explicit prerequisite and correct invocation pattern. It doesn't name alternatives or exclusions, but the reference to menu_list gives practical guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• page_listA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so read-only behavior is established. The description adds that it returns GIDs, handles, and publish status, and that the id is intended for subsequent updates, providing useful context beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the action, the key output fields, and cross-references page_update. No redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only list tool, this description is sufficient: it specifies the output fields and the downstream use case. It is complete given the tool's simplicity and the available annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty with 0 parameters, so baseline is 4. The description doesn't need to add parameter details; it clarifies the output contents instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists the store's pages and specifies exact fields (GIDs, handles, publish status). It differentiates from sibling list tools like menu_list by focusing on pages, and the verb 'List' with resource 'pages' is specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a concrete usage context: use the returned id for page_update. It doesn't explicitly mention when not to use it or alternatives, but the purpose is clear enough to distinguish from sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shop_get_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation covers the safety profile. The description adds value by listing the exact output fields (name, domain, currency, plan), giving the agent a concrete expectation of the return value. This is useful behavioral context beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and front-loaded, with no wasted words. It states the action and the key output fields in a compact format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool, the description is complete. It explains what information will be returned, and no output schema exists to shift the burden. The sibling context further clarifies its unique purpose within the toolset.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts no parameters, so the description has no parameter burden to carry. The baseline score of 4 applies because no additional parameter documentation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') with a clear resource ('basic info about the connected Shopify store') and enumerates the exact fields (name, domain, currency, plan). This distinguishes it from sibling tools that target pages, menus, or themes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: whenever shop-level basic info is needed. It does not explicitly mention alternatives, but the sibling tool names make the context unambiguous. Clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• theme_listA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint=true already signals a safe read operation, and the description adds value by disclosing the output content (GIDs and roles) and the special meaning of role MAIN. It does not mention pagination or limits, but for a simple list tool with no parameters this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences that are front-loaded with the primary action. Every sentence earns its place: the first defines the tool, the second explains the significance of role MAIN and cross-references sibling tools.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description clearly states what the tool returns (GIDs, roles) and provides the domain knowledge that MAIN is the live theme. It also links to downstream usage, making the tool's purpose and integration context fully understandable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so there is nothing to explain. The baseline for 0 params is 4, and the description does not need to add parameter details. It focuses on the output, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'list' with a clear resource ('the store's themes') and specifies what is included (GIDs, roles). It distinguishes itself from sibling tools like page_list and menu_list by scoping to themes, and even names the dependent tools theme_read_file and theme_update_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool: to obtain theme IDs for use with theme_read_file and theme_update_file. It also provides practical context about the role MAIN being the published live theme, which helps the agent understand which theme to select.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.