Server Quality Checklist

Tool Scores

• korral_get_order_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool's safety profile is clear. The description adds valuable behavioral context, explaining that it can be used to check upstream state after a write call returned WRITE_OUTCOME_UNKNOWN, which helps the agent decide whether to resubmit. This goes beyond the annotations by framing the tool's role in a workflow, though it does not describe return format or other edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose in the first sentence and usage guidance in the second. Every clause earns its place with relevant information, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with rich annotations and complete schema coverage, the description is highly complete. It explains the tool's role in the order lifecycle, when to use it, and how it relates to the write operation. The absence of an output schema is not a gap because annotations and usage guidance provide sufficient context for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions: order_id is 'Order ID returned by korral_raise_replenishment_order' and store_id specifies 'Which store the order belongs to... Optional if this deployment is authorized for exactly one store; required otherwise.' The description does not add additional parameter semantics beyond what the schema already provides, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Look up the status of a replenishment order previously raised with korral_raise_replenishment_order, by its order_id.' It uses a specific verb ('look up'), identifies the resource (replenishment order status), and mentions the key parameter (order_id). It also distinguishes from siblings by focusing on order status lookup versus listing stores, checking stock risk, listing sales, or raising orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this to follow up on an order from an earlier session, or to check upstream state after a write call returned WRITE_OUTCOME_UNKNOWN before deciding whether to resubmit.' This tells the agent when to use it, though it does not explicitly state when not to use it or name alternatives. Since the sibling tools serve different purposes, the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• korral_list_recent_salesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish safe read-only/idempotent behavior. The description adds concrete behavioral details: returns individual transaction lines (not aggregates), sorted most recent first, and serves as an audit trail. This goes beyond the schema's parameter descriptions. Minor gap: no mention of return fields or pagination, but schema covers limit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action, and the second sentence provides valuable selection context without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a medium-complexity read-only tool with well-documented schema and clear annotations, the description provides purpose, granularity, ordering, and use-case guidance. The only missing piece is what fields appear in each transaction, but no output schema exists and the description's clarity compensates. It's sufficiently complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline applies. The description reinforces that sku and store_id define the target, and context implies limit/since control the window, but it doesn't add new parameter-level meaning beyond the schema. It does tie 'sales velocity' to the sku parameter, which adds a hint, but not substantial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') with a clear resource ('individual POS transactions') and scope ('for a SKU at a store, most recent first'). It also differentiates from sibling tools by referencing korral_check_stock_risk and framing this as the transactional detail behind that aggregate view.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: to break down sales velocity into basket sizes or as an audit trail against projections. Names the related sibling tool (korral_check_stock_risk) and gives a concrete decision scenario, providing clear guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• korral_check_stock_riskA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, openWorld, idempotent, and non-destructive. The description adds valuable behavioral context: it states the tool 'combines current on-hand, recent sales velocity, and supplier lead time into one deterministic projection with an explicit confidence level' and notes that 'the evidence used is included so the projection can be checked without another call.' No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences. The first sentence front-loads the purpose, the second adds transparency and an explicit alternative. No filler or redundancy — every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description tells the agent what to expect: a deterministic projection, confidence level, data-quality caveats, and included evidence. It also orients the agent relative to sibling tools and explains the need for store authorization. The combination of annotations and description is sufficient for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, including details for store_id (optional only if one store is authorized) and lookback_days (default, max, exclusive minimum). The tool description itself does not add parameter meaning beyond the schema; it only mentions 'SKU at a store' in passing. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a very specific purpose: 'Assess whether a SKU at a store will run out before closing today, and whether a replenishment order would arrive in time.' This clearly identifies the verb (assess) and resource (stock risk for a SKU/store), and distinguishes it from siblings like korral_list_recent_sales by explaining what evidence it includes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly names korral_list_recent_sales as an alternative when the user needs individual transactions, and the store_id parameter in the schema points to korral_list_authorized_stores. This provides clear when-to-use and when-not-to-use guidance, exceeding the minimum.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• korral_list_authorized_storesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false. The description adds context beyond these: it specifies the output includes local time and closing time, and warns that sibling tools will reject store_ids not returned here, implying this list is authoritative for valid IDs. No contradictions. Minor gap: it doesn't specify the exact response structure, but for a no-param read-only list this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no wasted words. The purpose, usage guidance, and edge-case default are all front-loaded and clearly structured. Each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only list tool with strong annotations and no output schema, the description is complete: it states what is listed, what fields are included, the tool's role as a prerequisite, and the single-store default behavior. It covers both the output and the logical relationship with sibling tools, leaving no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the baseline is 4. The description adds value by explaining that the tool's output contains store_id values which are used as parameters for other tools, and clarifies the optionality/default behavior. This bridges the output to subsequent parameter usage, which is helpful even though this tool itself has no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('Korral stores this deployment is authorized to query'), with specific detail about returned data (local time, closing time). It distinguishes this tool from siblings as the discovery mechanism for valid store_id values, which none of the listed sibling tools provide.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call this tool first, explains that every other tool rejects invalid store_id values, and describes the default behavior when exactly one store is authorized. This gives unambiguous when-to-use guidance relative to all sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• korral_raise_replenishment_orderA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations offer only high-level hints (readOnly=false, idempotent=false). The description adds substantial behavioral detail: dry-run default with preview contents, confirm gating execution, ALLOW_WRITES requirement, and the critical caveat that confirm=true reflects agent judgment not human approval. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, then tightly packed with essential workflow and safety information. Every clause earns its place; no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description covers the return value preview elements (order, on-hand, sales velocity, stockout projection, warnings, sanity check), execution prerequisites, and a safety caveat. For a tool with this complexity, the description is fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description enriches parameter understanding by explaining confirm's dual dry-run/execute behavior, store_id optionality based on authorization, and idempotency_key's purpose—none of which are in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Raise a replenishment order for a SKU at a store.' It clearly distinguishes this write tool from sibling read-only tools like list_authorized_stores and check_stock_risk.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: dry run by default for human approval, confirm=true to execute, and ALLOW_WRITES requirement. It doesn't explicitly state when not to use it or name alternatives, but the write vs read contrast is implicit and the workflow is well described.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.