Server Quality Checklist

Tool Scores

• get_daily_pnlA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It usefully explains that COGS is resolved per line item and that refunded orders are excluded. However, it does not define what 'current order set' means or how the 'daily' period is determined, leaving ambiguity in the tool's scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the core metrics stated in the first sentence. Every subsequent line adds distinct value: cost-resolution method and refund exclusion. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema or annotations, so the description must communicate both the result metric and behavioral caveats, which it does adequately for a zero-parameter tool. The main gap is that the exact time period of 'current order set' is left undefined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema covers everything about parameters, so the description does not need to explain parameter semantics. It still adds meaning by describing what the computed values represent, which is sufficient given no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with the specific metrics 'Revenue, COGS, and gross profit' and names the resource 'current order set.' This clearly distinguishes the tool from its sibling get_orders, which would return orders rather than a financial summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives helpful context such as 'across the current order set' and 'Refunded orders are excluded,' but it does not explicitly state when to use this tool versus get_orders or when not to use it. Usage guidance is implied rather than clearly articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ordersA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It usefully discloses that pagination is handled automatically, which is a non-obvious behavior an agent needs to know. It does not mention auth or rate limits, but for a simple fetch operation the pagination disclosure is meaningful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences cover purpose, pagination behavior, and the parameter's allowed values. Every sentence earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter fetch tool with an output schema, this description is complete. It covers what the tool does, the parameter semantics, and a key behavioral detail (automatic pagination). The sibling tool is clearly distinct, so no additional routing context is necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It fully defines the only parameter, status, and lists all allowed values: 'any', 'open', 'closed', or 'cancelled'. This adds clear meaning beyond the schema, which only shows type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific action and resource: 'Fetch Shopify orders'. The automatic pagination note adds scope, and the sibling get_daily_pnl is clearly a different resource, so an agent can distinguish them without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when order data is needed and offers a status filter, but it does not explicitly say when to use this tool versus get_daily_pnl or any other alternative. There are no exclusions or conditional routing cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.