Server Quality Checklist

Tool Scores

• salesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses pagination, date format, default behavior for 'after', and output details (per-sale price, product, email domain, refund state, totals), providing useful behavioral context beyond a bare list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, information-dense sentences. Every phrase adds value: pagination, date format, default behavior, and return fields are all covered without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with no required parameters and an output schema, the description covers key aspects (pagination, date range, output fields). Minor ambiguity remains around 'totals' and how pagination is controlled, but overall it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are absent (0% coverage), so the description must compensate. It explains 'after' semantics (defaults to today) but leaves 'before' and 'product_id' vague, offering only partial parameter clarification.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists sales with a specific verb ('List') and resource ('sales'), distinguishes it from sibling tools (list_products, revenue_summary) by focusing on sales records, and adds important context about pagination and date filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving sales data and provides parameter guidance (e.g., 'after' defaults to today), but it does not explicitly state when to use this tool over siblings or mention exclusions/alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_productsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden. It clearly states this is a read-only listing operation and defines its scope ('all products in the store'), including the returned fields. It does not disclose pagination, ordering, or authorization, but for a simple zero-parameter list tool, the behavioral surface is adequately covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and every word adds value. The supporting 'answers...' phrase is concise and clarifies the tool's purpose without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple parameterless list tool with an output schema available. The description fully explains what data is returned and the intended use case. Sibling tools are financial summaries, so this catalog-focused description is complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100% trivially, so the baseline is 4. The description adds useful semantic value by explaining what the output represents, which is sufficient for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the operation as 'list all products' and specifies the exact fields returned (name, permalink, price, published state, sales count). The phrase 'what am I actually selling right now?' gives an outcome-oriented purpose that distinguishes this catalog tool from the sibling financial tools (sales, revenue_summary).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case ('what am I actually selling right now?') but does not explicitly state when to use this tool over the sibling sales and revenue_summary tools. It gives no exclusions or alternative references, so usage guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• revenue_summaryA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It discloses that it returns aggregate metrics, applies revenue filtering by `after` with a default, and distinguishes lifetime sales counts from date-filtered revenue. It does not cover error handling or auth, but for a simple read-oriented summary tool the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the tool's purpose, and every clause adds specific information. The only minor flourish is 'The number an operator actually wants first', which reinforces usage context without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and an output schema present, the description fully covers what the tool does, the key metrics, and the parameter's format/default. It is complete enough for an agent to select and invoke appropriately without further clarification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines `after` as a string with an empty default, providing 0% coverage. The description compensates by specifying the YYYY-MM-DD format and the default of 'start of current month', adding meaningful semantics that allow correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool as a one-call store health check, enumerating the exact metrics returned (product count, published count, lifetime sales counts per product, revenue since a date). It stands apart from siblings list_products and sales by providing an aggregated summary rather than raw lists or separate sales data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'One-call store health check' and 'The number an operator actually wants first' imply this is a quick aggregate alternative to calling multiple tools, but it does not explicitly name sibling tools or state when not to use it. The guidance is clear in context but lacks direct exclusions or alternative references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.