Server Quality Checklist

Tool Scores

• calculate_advancedB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions support for advanced features like multi-line scripts and control flow, which adds some context, but fails to describe critical behaviors such as error handling, execution limits, security implications, or what the output looks like (e.g., result format, potential side effects). For a tool executing scripts with no annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, stating the core purpose in the first clause. Both sentences add value by specifying capabilities (e.g., multi-line scripts, loops) without redundancy. However, it could be slightly more structured by explicitly separating features from usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing advanced scripts with no annotations and no output schema, the description is incomplete. It lacks information on return values, error conditions, execution constraints (e.g., timeouts, resource limits), and how it differs operationally from sibling tools. This leaves the agent with insufficient context to use the tool effectively in varied scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for both parameters in the input schema. The description adds minimal value beyond the schema by implying the 'script' parameter can include advanced constructs like loops and functions, but does not provide additional syntax or format details. With high schema coverage, the baseline score of 3 is appropriate as the schema does most of the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Execute advanced BC scripts with variables, functions, and control flow.' It specifies the verb ('execute') and resource ('advanced BC scripts'), and distinguishes it from the simpler 'calculate' sibling tool by mentioning advanced features like multi-line scripts, loops, and conditionals. However, it doesn't explicitly contrast with 'set_precision', which slightly limits differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating it 'Supports multi-line scripts, variable assignments, loops, and conditionals,' suggesting it should be used for complex calculations beyond basic arithmetic. However, it lacks explicit guidance on when to choose this tool over 'calculate' or 'set_precision', and does not mention any prerequisites or exclusions, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_precisionA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's effect ('affects all calculations until changed again') which is useful context, but doesn't mention potential side effects, error conditions, or what happens if precision is set to extreme values. The description doesn't contradict annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence states the core purpose, and the second explains the persistence effect. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter configuration tool with no annotations and no output schema, the description provides adequate but minimal context. It explains what the tool does and its persistence effect, but doesn't address potential limitations, error scenarios, or how this interacts with sibling calculation tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single 'precision' parameter with its type, range, and description. The description doesn't add any additional parameter semantics beyond what's in the schema, which meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Set') and resource ('default precision for subsequent calculations'). It explains what the tool does (sets decimal places for calculations) but doesn't explicitly differentiate from sibling tools like 'calculate' or 'calculate_advanced'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool ('for subsequent calculations') and mentions persistence ('affects all calculations until changed again'), but doesn't provide explicit guidance on when to use this versus the sibling calculation tools or any prerequisites for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculateA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the computational behavior (BC calculator, arbitrary precision arithmetic, supported operations) but doesn't mention error handling, performance characteristics, or limitations beyond the precision parameter. It provides adequate but not comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in a single sentence that front-loads the core purpose and then lists supported features. Every element (calculator type, precision capability, operation categories, function examples) serves a clear informational purpose with zero wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mathematical calculation tool with 2 parameters (1 required) and no output schema, the description provides good context about the calculator engine, precision capabilities, and supported operations/functions. However, it doesn't mention what the output looks like (numeric result format, error responses), which would be helpful given the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('evaluate') and resource ('mathematical expressions'), and distinguishes this tool from its sibling 'calculate_advanced' by specifying it uses BC calculator with arbitrary precision arithmetic and supports basic operations and math library functions. This provides immediate understanding of what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by listing supported operations and functions, but doesn't explicitly state when to use this tool versus 'calculate_advanced' or 'set_precision'. It provides functional scope but lacks explicit guidance on tool selection or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.