Server Quality Checklist

Tool Scores

• check_inventoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It states a simple check but does not describe whether the operation is read-only, what response format to expect (though an output schema exists), how invalid material IDs are handled, or any authentication requirements. The description adds little beyond the tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that directly states the tool's purpose with no redundant or irrelevant information. It is appropriately sized for a simple one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema, the description is minimally adequate. However, it lacks differentiation from sibling tools and provides no situational context, making it incomplete for an agent trying to select among similar inventory-related tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'specific material ID' but provides no additional meaning about the parameter, such as format, source, or constraints. Since schema description coverage is 0%, the description was expected to compensate, but it only restates the parameter name without clarifying its semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks inventory level for a specific material ID, which is a specific verb+resource. It inherently differentiates from list-style siblings like get_low_stock_items and get_open_orders, but does not explicitly name alternatives or scope limitations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the sibling tools. It does not mention any exclusions, prerequisites, or alternative tools, leaving the agent to infer its appropriate context from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_open_ordersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior but only states the basic retrieval action. It fails to mention pagination, default status behavior, or any side-effect or safety implications. A read operation is inferred, but this is not disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no fluff. It is concise, though slightly under-specified, which is acceptable for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description lacks critical context such as parameter semantics, default behavior, and relationship to sibling tools. It is a minimal viable description but leaves the agent to infer how to invoke it correctly, especially regarding 'status' values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only mentions 'status' but does not explain valid values or default, and it omits 'limit' entirely. The schema provides defaults but no semantic meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a list of sales orders by status, using a specific verb and resource. It distinguishes from all sibling tools which deal with inventory or requisitions, not sales orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives, nor does it mention exclusions or prerequisites. The only implied usage is that it's for sales orders, but no explicit context is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• approve_pending_requisitionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for transparency. It only discloses the human-trigger requirement, but does not mention side effects, irreversibility, permission requirements, or what the approval action entails. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences. The first states the purpose clearly, and the second emphasizes a critical human-in-the-loop constraint. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is an approval action with side effects, yet the description provides minimal context. It lacks prerequisites, postconditions, and any explanation of the approval workflow. The existence of an output schema may cover return values, but the overall description is insufficient for an AI agent to safely and correctly invoke the tool in a given context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no explanations for 'requisition_id' or 'approved_by'. The parameter names are somewhat self-explanatory, but the tool does not compensate for the lack of schema documentation, leaving meaning entirely to inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Approve a pending purchase requisition') with a specific verb and resource. It distinguishes itself from siblings like create_requisition and get_open_orders, and the extra note about human triggering adds clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage constraint: it MUST be triggered by a human, implying it should not be used in automated flows. However, it does not explicitly name alternatives or conditions under which this tool is preferred over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_requisitionA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses the key constraint that the agent cannot approve its own requisition, which is non-obvious and valuable. However, it does not mention other potential side effects such as validation behavior or permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and immediately states the core purpose. The second sentence adds crucial behavioral context without unnecessary words. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple create operation with 3 required parameters and an output schema, so return value details are likely covered. The description provides the essential workflow context (pending approval) and the critical self-approval restriction. It does not address error handling or preconditions, but given the presence of an output schema and sibling context, this is sufficient for a minimal viable description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for the parameters, and the description adds none either. The agent must rely solely on parameter names (material_id, quantity, requested_by), leading to ambiguity about required formats, constraints (e.g., positive quantity), or source of values. The description does not compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and the resource ('purchase requisition'), with a parenthetical clarifying its initial state ('pending approval'). This uniquely distinguishes it from sibling tools like approve_pending_requisition, which perform a different action on the same entity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates this tool is for creating requisitions but does not explicitly state when to use it over alternatives. The note about not being able to approve its own requisition implies a workflow dependency but offers no explicit usage guidance or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_low_stock_itemsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly states the tool is a listing operation (read-only) and defines the filter. It doesn't cover edge cases like empty results or pagination, but for a simple list tool, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and front-loaded with the primary action and target. Every word is necessary, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description fully specifies the tool's behavior. The list of all low-stock items is unambiguous, and no further context is required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is trivially complete. The description adds no parameter details because there are none to explain, aligning with the baseline score of 4 for no-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and identifies the exact resource (inventory items) and condition (quantity on hand below reorder point). This clearly distinguishes it from sibling tools like get_open_orders and check_inventory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the use case: when you need all items with low stock. It does not explicitly name alternatives or exclusions, but the context (sibling tools) makes the distinction clear enough, earning a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.