Server Quality Checklist

Tool Scores

• check_compatibilityB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It adequately states that the tool returns a boolean indicating dimensional compatibility, implying it is read-only with no side effects. However, it does not explicitly mention that it does not modify data or require special permissions, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two sentences, with no superfluous words. It front-loads the core action ('Check if two units are dimensionally compatible') and adds a clarifying parenthetical and return value explanation. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description provides a basic understanding but lacks completeness. It does not specify the format of unit strings or mention the output schema (which likely indicates the boolean return). For a tool with siblings, more context on when to use it would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions for the two required parameters (unit_a, unit_b), and the description does not provide any additional meaning about them. It fails to explain that these are strings representing unit expressions, or what format is expected. With 0% schema coverage, the description offers no parameter-level guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool checks if two units are dimensionally compatible, with the elaboration that it checks if they can be converted. This is distinct from sibling tools like 'convert' (which performs conversion) and 'list_compatible_units' (which lists all compatible units), making the purpose highly specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus its siblings. For example, it does not suggest using this tool before calling 'convert' to verify compatibility, nor does it mention that 'list_compatible_units' could provide a list of compatible units. The description gives no context on appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convertA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description discloses return value shape (converted value, conversion factor, or structured error for incompatible units), which is helpful. No annotations are present, so the description carries the burden. It does not mention idempotency or side effects, but for a conversion tool, the behavior is well captured.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The key information (what it does, what it returns) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of unit conversions (multiple unit systems, compound units), the description is too sparse. Parameter details are missing, and there is no cross-reference to sibling tools that might help with unit discovery. The output schema may exist but is not available to evaluate; still, the description does not fully compensate for the lack of schema descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage, so the description must compensate. However, the description does not explain what from_unit and to_unit expect (e.g., string format, allowed values). It only says 'unit' without details, leaving the agent guessing about accepted unit syntax.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb ('convert') and resource ('value from one unit to another'), and distinguishes from siblings like check_compatibility (which only checks) and list_compatible_units (which lists).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or not-to-use advice, but the tool's purpose is clear enough that it is the main conversion tool. Sibling tools are conceptually distinct, but no guidance is given on when to choose convert over check_compatibility or others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_quantityA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It discloses the return structure but does not mention error handling, edge cases, or any side effects. For a pure parsing function, the behavioral profile is partially covered, but there is room for more transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with the purpose stated first, followed by examples and return details. Every sentence adds value, and there is no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema available), the description covers the core functionality and return values. It adds context about dimensionality and SI equivalent not in the schema. However, it lacks notes on error handling or input validation, but the output schema likely fills return structure gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, requiring the description to compensate. It provides examples ('100 mg/L', '9.81 m/s²') which add practical meaning, but does not formally define the expected format, constraints, or allowed syntax for the expression parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool parses a quantity string into structured components, with examples and a list of returned fields (magnitude, units, dimensionality, SI equivalent). This is a specific verb+resource and distinguishes from sibling tools like convert or check_compatibility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to parse a quantity string but provides no explicit guidance on when to use this tool versus alternatives (e.g., convert, list_compatible_units). No when-not-to-use or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simplifyA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behaviors: adjusting prefixes, reducing named units, and providing base SI. However, it could be more explicit about edge cases or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences totaling about 20 words, efficient and front-loaded with no superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and the presence of an output schema, the description adequately covers the main action. It explains what simplification entails and mentions base SI representation, but lacks details on valid input formats or handling of complex expressions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter 'expression' exists, and schema description coverage is 0%. The description simply refers to 'a unit expression' without adding details on format, valid units, or structure, providing marginal value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'simplify' and the resource 'unit expression', specifying it adjusts prefixes, reduces named units, and provides base SI representation. This distinguishes it from siblings like 'convert' and 'check_compatibility'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context but does not explicitly state when to use this tool versus alternatives like 'convert' or 'check_compatibility'. No when-not-to-use or prerequisite information is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_compatible_unitsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It reveals that the tool returns all canonical unit names of the same physical dimension, including imperial and US customary units. It does not cover error handling or edge cases, but for a simple listing tool, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short, focused sentences. It is front-loaded with the purpose and provides all necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, existence of output schema), the description adequately covers the tool's purpose, input, and output semantics. No additional information is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'unit' has no schema description (0% coverage). The tool description adds meaning by stating it lists units compatible with the given unit, indicating the parameter is the unit to check. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all units compatible with a given unit, specifying it returns canonical unit names sharing the same physical dimension, including imperial and US customary. This is a specific verb+resource, and distinguishes from sibling tools like 'convert' or 'check_compatibility'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (to list all compatible units), but does not explicitly state when not to use it or provide direct comparisons to siblings. Given the context, an agent can infer usage, but explicit guidance would improve the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.