Server Quality Checklist

Tool Scores

• calculateB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden. It mentions 'safely' and specifies that log is base 10 and ln is natural, which adds behavioral context beyond the schema. However, it does not disclose error handling, return type, or limitations (e.g., no complex numbers). Given the absence of annotations, the description provides some transparency but lacks depth on behavior in edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description opens with the primary purpose and then efficiently lists supported operations. It is not overly verbose, and the list is necessary to inform the agent of capabilities. The structure is clear and front-loads the key information. It earns a 4 because the list could be seen as a bit dense, but it is appropriately concise for a calculator tool with many operations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the supported operations and variables, which is adequate for most use cases. However, it does not mention the output format (e.g., returns a number) or error handling for invalid expressions. Given the complexity of the tool (many functions), a note about return values or potential errors would make it more complete. The list of functions is comprehensive, but the lack of detail on edge cases leaves it slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description covers both parameters at 100%, so the baseline is 3. The description mentions support for named variables but does not add semantics beyond the schema's examples. It lists supported functions but these relate to the expression format, not the parameter definitions. No additional meaning is provided, so it remains at the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool evaluates mathematical expressions and lists the supported operators and functions. It does not explicitly differentiate from siblings like convert_units or statistics, but the general-purpose nature is evident from the list of operations. It is specific enough for an agent to understand its core function, though not as exemplary as explicitly naming a sibling to contrast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives. The description implies it is for arbitrary math evaluation, but it does not state exclusions or refer to sibling tools. An agent can infer usage from the purpose, but without explicit alternatives, it falls short of providing clear decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• percentage_calcA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden. It does explain the semantics of each operation (e.g., percent_of, tip, discount), which is helpful. However, it does not disclose return format, rounding behavior, edge cases like zero values, or error handling, which a calculator tool would benefit from stating.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence that front-loads the tool's purpose and then concisely lists every operation with its relevant inputs. There is no filler or repetition, and it efficiently conveys a lot of operational detail in a small space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all operations and parameter mappings, and the schema provides rich field descriptions, so the main calling contract is clear. However, there is no output schema and the description does not mention what the tool returns (e.g., a single number, a per-person split amount, or an object), which is a notable gap for an agent executing the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is already 3. The description adds meaningful operational context by mapping parameters to each operation, such as 'tip (bill + tip% + optional split)' and 'percentage_change (old to new)', clarifying which parameters apply to which calculation. This goes beyond the individual field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs common percentage calculations and enumerates six specific operations with their formulas. It is unambiguous about what the tool does, but it does not explicitly distinguish itself from sibling tools like calculate or statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by listing the supported operations and their parameter mappings, so an agent can infer when to use it for percentage tasks. However, it gives no explicit guidance about when not to use it or when a sibling tool like calculate or statistics would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• statisticsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description must carry the full burden of behavioral disclosure. It does reveal operation-specific parameter requirements, which is helpful, but it omits important behaviors such as the existence of the 'summary' operation, the meaning of the 'population' parameter (sample vs population variance), return value structure, and edge-case handling (e.g., empty datasets). This is a moderate amount of transparency but leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded with the core purpose and operation list. It avoids fluff and uses clear phrasing. However, it omits the 'summary' operation from the list, which is present in the schema enum, making the description slightly incomplete but not wordy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no output schema, and no annotations, the description provides enough to use the main operations but misses key aspects: 'summary' is not mentioned, 'population' behavior is not explained, and there is no indication of return format or edge cases. The description is adequate for basic use but not fully complete for an agent to handle all scenarios correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides detailed descriptions for all parameters (100% coverage). The description adds minimal value beyond this: it restates that percentile needs the 'percentile' parameter and correlation needs 'dataY', which the schema already documents. It does not clarify the 'population' parameter or the 'summary' operation. With high schema coverage, the baseline is 3, and the description provides no extra semantic depth.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Calculate statistics on a dataset' and enumerates the supported operations (mean, median, mode, etc.), making the tool's purpose unambiguous and distinct from generic calculators like 'calculate' or 'percentage_calc'. It also provides specific operation names, so an agent can immediately understand what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers some internal guidance (e.g., 'For percentile provide the percentile parameter', 'For correlation provide dataY'), but it does not explicitly state when to use this tool versus the sibling tools (calculate, convert_units, format_number, percentage_calc). There is no exclusion rule or alternative routing, so the when-to-use context is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• format_numberA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The description discloses the supported formats but does not state that this is a pure/read-only transformation, what it returns (a string), or any error behavior. It is not misleading, but it leaves basic behavioral aspects unstated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action ('Format a number') and immediately provides concrete examples of each format. There is no redundant or filler content; every word contributes to clarifying the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description, combined with the fully-documented schema, covers all supported formats and parameter roles. It does not explicitly state the return type, but this is strongly implied by the formatting intent. No critical information for calling the tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds illustrative examples (e.g., $1,234.56) but does not add meaning beyond the schema's parameter descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (format) and resource (a number) and enumerates the supported formats (currency, percentage, scientific, binary, octal, hexadecimal, significant figures, thousands separator), which clearly distinguishes it from sibling tools like calculate or convert_units.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for formatting numbers but does not explicitly state when to use it over percentage_calc or other siblings. There is no mention of exclusions or conditions. For example, when should an agent choose format_number with format='percentage' rather than percentage_calc? This is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_unitsA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It states the core action and supported units but does not mention return format, error handling for unsupported units, precision/rounding behavior, or any side effects. An agent knows what it converts but not what the response looks like or how failures are surfaced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence followed by a structured list of categories and units. It is compact, scannable, and front-loaded with the primary action. Every part earns its place; the unit list is dense but necessary for correct invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple conversion tool, the description covers supported units and categories, which is the critical information for calling it correctly. It lacks an output schema and does not explain the return type, but the expected output is easily inferred as the converted number. The absence of edge-case details (e.g., temperature offset formulas) is a minor gap, not a blocker.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema describes 'from' and 'to' with only single examples (e.g., 'mi', 'kg'), leaving the full set of valid units implicit. The description compensates by exhaustively listing all supported units per category, giving the agent complete vocabulary for these parameters. This adds meaningful semantic value beyond the schema's terse examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Convert a value between units,' a clear verb-resource pair that immediately identifies the tool's function. It then enumerates supported categories and their units, which unambiguously distinguishes it from the sibling tools (calculate, statistics, format_number, percentage_calc) that handle arithmetic rather than unit conversion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool—when a value needs converting between the listed unit categories. It does not explicitly name alternative tools or state when not to use it, but the context is clear enough given the distinct sibling set. There is no exclusions or 'use X instead' guidance, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.