Server Quality Checklist

Tool Scores

• weather__get_weatherC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions an HTTP call, hinting at network behavior, but doesn't disclose critical traits like error handling, rate limits, authentication needs, or what happens if the location is invalid. This leaves significant gaps for a tool that likely interacts with an external API.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action ('Get weather for location') and adds a useful detail ('via HTTP call'). However, it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (an HTTP-based weather tool with no output schema and no annotations), the description is incomplete. It lacks details on what weather data is returned, how location is specified (implied but not stated), error cases, or behavioral constraints. This makes it inadequate for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameters need documentation. The description doesn't add param info, but that's acceptable here. Baseline is 4 for zero parameters, as the schema fully covers the absence of inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose ('Get weather for location') and the mechanism ('via HTTP call'), which is clear but vague. It doesn't specify what weather data is retrieved (e.g., temperature, conditions) or how the location is determined, and it doesn't distinguish from siblings (calculator tools), though they are unrelated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, context for location input, or comparisons to other weather-related tools (none listed as siblings, but this is a generic gap). Usage is implied only by the purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculator__addB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Add two numbers' implies a simple computation, but it doesn't disclose any behavioral traits such as error handling, input validation, or performance characteristics. This leaves significant gaps in understanding how the tool behaves beyond its basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words, front-loading the core action ('Add') without any wasted text. Every word earns its place by directly conveying the tool's purpose, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (a simple addition function) and no output schema, the description is minimally complete. It states what the tool does but lacks details on usage, behavior, or return values. For such a straightforward tool, this might be adequate, but it doesn't provide full context for reliable agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description 'Add two numbers' implies two numeric inputs, adding semantic meaning about what the tool expects, which compensates for the lack of schema parameters. However, it doesn't specify parameter names or formats, so it's not fully detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Add two numbers' clearly states the tool's purpose with a specific verb ('Add') and resource ('two numbers'), making it immediately understandable. However, it doesn't explicitly distinguish this from its sibling 'calculator__multiply', which performs a different arithmetic operation, so it doesn't fully achieve sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention the sibling 'calculator__multiply' for multiplication needs or 'weather__get_weather' for unrelated tasks, nor does it specify any context or prerequisites for addition operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculator__multiplyB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Multiply two numbers' implies a computational operation but doesn't specify behavioral traits like error handling (e.g., overflow), input constraints (e.g., numeric types), or output format. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with 'Multiply two numbers,' a single sentence that front-loads the core purpose without any wasted words. Every part of the sentence earns its place by clearly stating the action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (a simple multiplication operation) and no output schema, the description is minimally complete but lacks details on behavior and usage. Without annotations or output schema, it should provide more context on how the tool works and what it returns, but it's adequate for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are defined in the schema. The description mentions 'two numbers,' which implies two inputs, but since the schema explicitly defines no properties, this is a minor semantic addition. Baseline is 4 for 0 parameters, as there's nothing for the description to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Multiply two numbers' clearly states the verb ('multiply') and resource ('two numbers'), making the purpose immediately understandable. It distinguishes from sibling tools like 'calculator__add' by specifying multiplication rather than addition. However, it doesn't explicitly differentiate from other potential mathematical operations beyond the named siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when multiplication is appropriate compared to addition or other operations, nor does it reference the sibling tools or any contextual prerequisites. Usage is implied by the tool name but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.