Server Quality Checklist

Tool Scores

• create_templateC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states 'Create' which implies a write/mutation operation, but doesn't disclose behavioral traits like permissions needed, whether it's idempotent, error handling, or what happens on success/failure. For a creation tool with complex inputs, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. However, it's too brief given the tool's complexity—it could benefit from more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with 5 parameters (including nested objects), no annotations, and no output schema, the description is incomplete. It doesn't explain the template's purpose, what happens after creation, or error cases, leaving significant gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by mentioning 'shop domain' but doesn't explain parameter relationships or usage beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('Create a new template') and mentions the resource ('template'), but it's vague about what a template is and doesn't differentiate from sibling tools like 'get_list_templates' or 'get_detail_template'. It mentions 'with shop domain' but doesn't explain why that matters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'get_list_templates' for viewing templates. The description doesn't mention prerequisites, context, or exclusions, leaving the agent with no usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_detail_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves a template but doesn't describe what 'detail' includes, whether it's a read-only operation, potential error conditions, or the response format. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose and parameters without any unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is insufficient for a tool that retrieves data. It doesn't explain what 'detail' entails, the return format, or any behavioral aspects like error handling. For a read operation with no structured output documentation, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions the two parameters ('template id and shop domain'), but the input schema already provides 100% coverage with clear descriptions for both. The description adds no additional meaning beyond what's in the schema, such as format examples or constraints, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get detail template') and identifies the required resources ('template id and shop domain'), making the purpose understandable. However, it doesn't differentiate this tool from its sibling 'get_list_templates' (which presumably lists templates rather than retrieving details of a specific one), preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_list_templates' or 'create_template'. It mentions the required parameters but doesn't explain the context or prerequisites for invoking this tool, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_list_layers_of_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what the tool does without behavioral details. It doesn't disclose if this is a read-only operation, potential side effects, error handling, or rate limits, leaving significant gaps for a tool that likely queries data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. It avoids unnecessary words, though it could be slightly more structured by separating purpose from parameter mentions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It lacks details on return values (e.g., what 'layers' entail), error conditions, or behavioral traits, which are critical for a tool with two required parameters and likely data retrieval functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds no additional meaning beyond what's in the schema (e.g., it repeats that template id and shop domain are needed without extra context), meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get list layers') and the resource ('template'), specifying it requires template id and shop domain. It distinguishes from siblings like 'get_detail_template' by focusing on layers rather than template details, but doesn't explicitly contrast with 'get_list_templates' which might list templates rather than layers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. The description mentions required parameters but doesn't specify use cases, prerequisites, or exclusions compared to sibling tools like 'get_detail_template' or 'create_template'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_list_templatesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the required 'shop domain' context but fails to describe critical behaviors: whether this is a read-only operation, if it requires authentication, how pagination works with 'limit' and 'page', or what the return format looks like. For a list retrieval tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and includes essential context ('with shop domain'). Every part earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no output schema, no annotations), the description is incomplete. It lacks information on return values, error handling, authentication needs, and how parameters like 'filter' and 'sort' function. Without annotations or output schema, the agent has insufficient context to use the tool effectively beyond basic parameter passing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds minimal value beyond implying that 'shopDomain' is required (though it's not marked as required in the schema). It doesn't explain parameter interactions or provide additional context beyond what's in the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('list templates'), making the purpose understandable. It specifies the required context ('with shop domain'), which helps differentiate it from generic template retrieval tools. However, it doesn't explicitly distinguish from sibling tools like 'get_detail_template' or 'get_list_layers_of_template', preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_detail_template' for single templates or 'create_template' for creation, nor does it specify prerequisites or exclusions. The agent must infer usage from the name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.