Server Quality Checklist

Tool Scores

• get_all_customersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions automatic store hash retrieval from environment variables, which is useful context, but lacks details on permissions, rate limits, pagination behavior (beyond parameters), error handling, or what the return format looks like. For a read operation with 19 parameters, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads key information (getting customers with filtering). It could be slightly more structured by separating the automatic store hash note, but it avoids redundancy and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (19 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain the return format, error cases, or behavioral traits like pagination limits or authentication needs. The automatic store hash note is helpful, but overall, it falls short for a tool with many parameters and no structured output guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 19 parameters thoroughly. The description adds minimal value by listing some filter types (email, name, company, phone, customer group, dates, pagination) but doesn't provide additional syntax, format, or usage context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('all customers from the BigCommerce API'), making the purpose evident. However, it doesn't explicitly differentiate from sibling tools like 'get_all_orders' or 'get_all_products' beyond mentioning customers specifically, which is implied but not contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'comprehensive filtering options' and automatic store hash retrieval, but provides no explicit guidance on when to use this tool versus alternatives (e.g., for filtering vs. other customer-related tools). There's no mention of prerequisites, exclusions, or sibling tool comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_productsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions automatic retrieval of store hash from environment variables, which is useful context, but doesn't describe important behavioral aspects like pagination, rate limits, authentication requirements, error conditions, or what 'all products' means in practice (e.g., maximum results, filtering options).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that both add value. The first sentence states the core purpose, and the second provides important implementation context about environment variable usage. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what the tool returns (product format, data structure), doesn't mention pagination or result limitations for 'all products,' and provides minimal behavioral context. The agent would struggle to use this tool effectively without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single optional parameter. The description adds the context that store hash is automatically retrieved from environment variables when not provided, which provides useful operational context beyond the schema's technical documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('all products from the BigCommerce API'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like get_all_customers and get_all_orders, but the resource specificity provides implicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools, prerequisites, or contextual factors that would help an agent decide between get_all_products, get_all_customers, or get_all_orders.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_ordersB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that store hash is automatically retrieved from environment variables, which is useful context about configuration. However, it doesn't describe critical behavioral traits like whether this is a read-only operation, pagination behavior (implied by limit/page parameters but not explained), rate limits, authentication requirements, or what happens when no filters are applied. For a tool with 19 parameters and no annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with three sentences that each serve a purpose: stating the core function, explaining a key filtering use case, and providing implementation detail about store hash. It's front-loaded with the main purpose and avoids unnecessary elaboration. However, the second sentence about customer_id filtering could be more tightly integrated with the first sentence for better flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (19 parameters, no annotations, no output schema), the description provides a basic but incomplete picture. It covers the core purpose and one filtering scenario but doesn't address the tool's full behavioral context, return format, error conditions, or relationship to sibling tools. The 100% schema coverage helps with parameter understanding, but the description alone doesn't provide enough context for confident agent usage without additional inference from the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 19 parameters thoroughly. The description adds minimal value beyond the schema by mentioning customer_id filtering specifically and noting that store hash can be auto-retrieved from environment variables. This provides some contextual meaning but doesn't significantly enhance understanding beyond what's already in the parameter descriptions. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get all orders from the BigCommerce API.' It specifies the resource (orders) and the action (get), though it doesn't explicitly differentiate from sibling tools like get_all_customers or get_all_products beyond mentioning different resources. The mention of filtering by customer_id adds some specificity but doesn't fully distinguish it from potential order-related alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by mentioning filtering by customer_id to get products associated with specific customers, but it doesn't explicitly state when to use this tool versus alternatives or any prerequisites. The note about store hash being automatically retrieved from environment variables offers some operational guidance, but no explicit when/when-not instructions or sibling tool comparisons are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.