Server Quality Checklist

Tool Scores

• get_customerB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations and the description does not mention side effects, read-only behavior, permissions, or potential errors, so the agent is not informed about behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no unnecessary words or repetition, making it easy to parse and act on.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple lookup but does not specify what 'customer details' includes or what the response shape will be, especially since no output schema is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The customer_id parameter is only described as 'customer ID', adding little beyond the schema title. With 0% schema description coverage, the description should provide more detail about the parameter's format or meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('customer details'), and the required input ('customer ID'), distinguishing it from arithmetic tools and the balance-specific sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus get_customer_balance or find_customer, leaving the agent to infer the appropriate choice from the tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_customerB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations and the description does not mention whether the operation is read-only, what it returns, or how it handles missing or ambiguous names. Since 'find' suggests a lookup, the lack of explicit side-effect or result information leaves uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the tool's purpose and key parameter. No unnecessary words or redundant details are present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool, the description gives the essential action and input, but it omits output shape and error behavior. Without an output schema, more detail about what is returned would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, 'name', is mentioned in the description as the lookup key, so its basic purpose is clear. However, the description does not clarify exact matching, case sensitivity, or whether multiple matches are possible.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds a customer using their name, which identifies both the action and the resource. It is distinguishable from sibling tools like get_customer and get_customer_balance, though 'find' is slightly less precise than 'get'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'using their name' implies the intended use when a customer name is available. However, it does not explicitly contrast with get_customer or explain when to prefer this tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• multiplyA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations to supplement the description, so the full burden falls on the text. It does not disclose return type, error behavior, or any side effects, leaving the agent to assume the operation is pure and returns a numeric result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no extraneous information. It is highly concise and directly conveys the operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and the presence of an output schema, the description is adequate for an agent to understand the operation. It could mention the result type, but the schema likely covers that, so nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'two numbers' which maps directly to the parameters a and b, but it does not elaborate beyond that. The schema defines them as integers, and the description adds minimal semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('multiply') and the resource ('two numbers'), leaving no ambiguity about the tool's purpose. It distinguishes itself from the sibling tools add and subtract by specifying the multiplication operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description directly implies the use case (performing multiplication), but it does not explicitly state when to choose this tool over alternatives like add or subtract. The usage is inferred from the verb, but no conditional guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_customer_balanceA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The word 'get' implies a read-only operation, but the description does not explicitly disclose side effects, error behavior (e.g., customer not found), or output format. With no annotations, the description carries the full burden and only partially covers transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no redundancy or unnecessary details. It efficiently conveys the core functionality without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter, the description provides enough context to understand the primary purpose. However, it does not mention potential error cases, return value specifics, or interaction with sibling tools, leaving minor gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, customer_id, is self-explanatory by name, but the schema description coverage is 0%, and the tool description adds no clarifying details about its meaning, constraints, or typical usage. Since coverage is low, the description should have compensated but did not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving a customer's balance. It distinguishes from sibling tools like get_customer and find_customer by focusing specifically on balance retrieval, making the action unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly explain when to use this tool versus the sibling tools. While the purpose is clear, it lacks guidance on scenarios where this tool is preferred over alternatives, such as when only balance is needed rather than full customer details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• subtractA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It accurately describes the core behavior without mentioning side effects, which is acceptable for a pure arithmetic function. However, it does not explicitly confirm the absence of side effects or describe return behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no extraneous details. It effectively communicates the core operation in minimal words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple arithmetic tool, the description is mostly adequate, but it lacks critical parameter details (order of operands) and any mention of return format or errors. The minimal schema and absence of annotations increase the need for a more thorough description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not clarify the order of subtraction. The phrase 'subtract one number from another' is ambiguous regarding whether a - b or b - a is intended. Since subtraction is non-commutative, this is a significant gap for the two integer parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb 'subtract' and the object 'one number from another', making it distinct from sibling tools like add and multiply.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description is self-explanatory for a simple arithmetic operation, but it does not explicitly state when to use this tool versus the sibling add or multiply tools. The scoping is implied by the operation rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• addA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The behavior is transparent for a pure arithmetic operation: it takes two numbers and returns their sum. It does not describe edge cases or return details, but the operation is simple and output schema is available.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no unnecessary words. It is well-structured and immediately understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple arithmetic tool, the description is complete enough for correct use. It lacks explicit edge-case or return-type details, but the operation's simplicity and available output schema reduce the need for further explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description says 'two numbers,' which collectively defines both parameters as addends. It does not individually describe a or b, but for addition the individual roles are clear and order is irrelevant.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's exact purpose: adding two numbers together. It is a specific verb and resource, and it naturally distinguishes from sibling tools like subtract and multiply.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes it clear that this tool is for addition, and sibling names imply when to use other operations. However, it does not explicitly state when not to use this tool or provide alternative selection guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.