Server Quality Checklist

Tool Scores

• create_replenishment_orderA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden and does well: it discloses validation, grouping by supplier, computing totals, and updating on-order quantities. This covers key mutation behaviors, though it omits details on error handling or rollback.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of three sentences, front-loading the purpose. Every sentence provides unique information without repetition, achieving high density of useful content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no output schema, the description covers core actions and mentions a 'confirmation with per-PO detail'. However, it lacks specifics on return format (e.g., order ID), error scenarios, prerequisites (store/SKU existence), and rate limits, leaving moderate gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds context like 'validates each SKU against the store' and 'groups lines by supplier', which are behavioral rather than parameter-specific. It does not enhance individual parameter meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Place' and resource 'replenishment (restock) order for one or more SKUs at a store'. It also details specific actions like validation, grouping, and updating, making it distinct from sibling tools like evaluate_replenishment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (to place a restock order) and distinguishes it from siblings by focusing on creation vs. evaluation/inventory retrieval. However, it does not explicitly state when not to use or provide alternative scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_replenishmentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral disclosure. It explains the evaluation logic, conditional order placement, default threshold, and safety of dryRun mode. It does not mention reversibility or side effects of placed orders, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: four sentences covering purpose, logic, output, and dryRun. It is front-loaded with the key verb and resource, and every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (evaluation + order creation, 4 parameters, no output schema), the description is remarkably complete. It explains inputs, process, conditional behavior, default, and dryRun. The sibling tools provide further context for alternative actions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds significant meaning beyond the schema by explaining the shortfall calculation (gap = last24h sales - on-hand), the role of gapThreshold, and dryRun as a safety toggle. This provides the agent with a holistic understanding of how parameters interact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool's purpose: evaluate a single SKU across one or more stores, compute shortfall based on last 24h sales vs. on-hand, and automatically raise replenishment orders. It distinguishes itself from siblings by specifying the combined evaluation and ordering action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (for evaluating low stock and ordering), mentions a default threshold, and highlights the dryRun option for safe testing. It does not explicitly state when not to use it, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_store_inventory_and_salesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly describes the output but does not explicitly state that the tool is read-only or has no side effects. However, the verb 'get' and the nature of the output (inventory/sales data) strongly imply it is non-destructive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence states purpose and main outputs, the second sentence specifies options. Information is front-loaded and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, the description compensates by listing six specific output fields (on-hand/on-order stock, 30-day sales velocity, revenue, days-of-supply, low-stock flags, suggested reorders). Together with parameter details, this provides a complete picture for an agent to understand what data the tool returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for three parameters. The description adds meaning by explaining the optional parameters: 'filter by category or show only low-stock items'. This goes beyond the schema descriptions, which are already clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a consolidated view of inventory levels and sales performance, listing specific output fields (per-SKU stock, sales velocity, revenue, etc.) and optional filters. This distinguishes it from siblings like create_replenishment_order and evaluate_replenishment which are action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for data retrieval to inform replenishment decisions, and the sibling tool names ('create_replenishment_order', 'evaluate_replenishment') reinforce the context. However, it lacks explicit guidance on when to use this vs. alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.