Server Quality Checklist

Tool Scores

• lookup_orderA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It adds an authorization constraint ('when the customer is authorized') and mentions 'customer-safe' output, which is useful. However, it does not describe what happens for unauthorized access, nonexistent orders, or confirm explicitly that the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with Args and Returns sections. Every sentence adds value, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter lookup with an existing output schema, the description covers the essential context: what is returned (customer-safe order status and delivery info), the authorization requirement, and parameter examples. It omits failure modes, but the output schema likely handles return details, and the tool is straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It provides example formats (ORD-1002, CUS-002) and clarifies that customer_id is the 'authenticated customer identifier,' adding meaning beyond the bare schema. It could be more explicit about the relationship between the two parameters but is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with the verb 'Look up' and the resource 'an order belonging to an authenticated customer.' It distinguishes itself from siblings like list_customer_orders (which lists multiple orders) and cancel_order (which modifies state) by focusing on a single order lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given for when to use this tool versus alternatives. The description does not mention that this is for retrieving a specific order by ID when the customer is known, nor does it reference sibling tools such as list_customer_orders for batch scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_account_statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It adds the important fact that the tool does not share local AuthContext and that the client is responsible for authentication/authorization. However, it omits details like whether the operation is read-only, error behavior, or rate limits, so transparency is only partial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the purpose. The additional two sentences about AuthContext are relevant and add necessary context without being overly verbose. Structure is clean and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema present, the description provides adequate context: it explains the purpose, the parameter's role, and the auth responsibility. It does not describe return values, but that is covered by the output schema. Some details like error conditions or example usage are missing, but overall it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It effectively explains the meaning of customer_id by noting that external MCP clients must pass it because AuthContext is not shared. This adds rationale beyond the schema's bare type definition, though it doesn't specify format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Retrieve the current status of a customer account.' This clearly distinguishes it from sibling tools (orders, product availability, policies) and leaves no ambiguity about its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool by explaining that it accepts a customer ID because an external MCP client doesn't share AuthContext. However, it does not explicitly mention alternatives or conditions for when to prefer this tool over siblings, leaving usage guidance somewhat implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_product_availabilityA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explains that it checks availability and returns stock quantity, which implies a read operation, but it does not disclose error handling, behavior when product is not found, or any side effects. This is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences: the first states the core function, the second gives usage context, and the third states the required parameter. No fluff or repeated information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema present, the description covers the purpose and when to use it. It lacks mention of edge cases or fallback behavior, but the simplicity and output schema lower the burden. It is nearly complete for its scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 0%, so the description must compensate. It only repeats that 'The product ID is required', adding no meaning about what the product ID represents, its format, or how to obtain it. This is redundant with the schema's required field.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('check'), a specific resource ('product'), and the result ('available stock quantity'). It is easily distinguishable from sibling tools like check_account_status or list_customer_orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when a customer asks whether a particular product is in stock', which provides clear context. It does not mention alternatives or exclusions, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_customer_ordersA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the operation as 'List' which implies read-only, but does not explicitly confirm no side effects, error behavior, or data safety. For a simple list operation, this is minimally adequate but lacks explicit transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: first delivers the core purpose, second provides usage context, third emphasizes a requirement. No redundant words and information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter list operation with an output schema, the description covers the main use case and gives usage context. It could mention what happens when no orders exist or the customer is not found, but overall it is sufficiently complete given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage, and the description only repeats that customer_id is required, which is already in the schema. It does not explain the format, domain, or source of the customer ID, adding minimal semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool lists orders associated with a specific customer, using a specific verb ('List') and resource ('orders'). It also distinguishes from sibling tools like lookup_order by noting it's for when no specific order ID is provided.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use this when the customer wants to view their orders but has not provided one specific order ID.' This provides a clear use case and implicitly excludes lookup_order, offering strong guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_policiesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It explicitly describes the return value ('matching policy chunks') and the fallback behavior ('grounded=False when nothing relevant enough was retrieved'). This adds useful transparency beyond what a search tool would infer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-organized into Args and Returns sections. Every sentence adds value, and there is no redundant or vague phrasing. It is a model of efficient documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter and existing output schema, the description is complete. It explains the purpose, the query construction, and the return behavior, including the grounded flag. No critical context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines 'query' as a string, but the description compensates with detailed guidance: 'A self-contained search query. Conversation history is not available on this surface, so include whatever context the search needs directly in the query text.' This adds significant semantic meaning beyond the schema, so the parameter is fully explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search company policy documents for text relevant to a question.' It uses a specific verb (search) and resource (policy documents) and distinguishes from sibling tools that handle customer orders and accounts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it tells the agent to craft a self-contained query because conversation history is unavailable. While it doesn't explicitly name alternatives, the purpose and sibling list imply this is for policy searches only, so this counts as clear context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_orderA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden of disclosing behavior. It clearly states this is a state-changing operation, explains the confirmation flow prevents accidental cancellation, and explicitly notes the authentication responsibility. It also indirectly signals irreversibility by contrasting 'preview' with 'without cancelling anything.'

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise opening statement, then a logical flow covering preconditions, the two-step confirmation process, and the client's responsibility. Each sentence earns its place, and the formatting (paragraph breaks) aids readability without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive state change, conditional gating, confirmation flow), the description covers all critical aspects: preconditions, cancellation states, preview/confirm workflow, and security responsibility. The presence of an output schema means return values need not be described, so the description is complete for safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though the input schema has no parameter descriptions (0% coverage), the description explains the semantics of all three parameters: order_id must exist and be cancellable, customer_id must own the order, and confirmed controls preview vs. actual cancellation. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb+resource: 'Cancel an order belonging to a specific customer.' It immediately distinguishes from sibling tools like lookup_order and list_customer_orders by emphasizing the cancellation action and the ownership constraint, leaving no ambiguity about the tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: the order must exist, belong to customer_id, and be in a cancellable state (Pending or Processing). It also prescribes the two-step workflow (preview with confirmed=False, then confirm with confirmed=True) and clarifies that the client must handle customer authentication. This is strong, actionable usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.