Server Quality Checklist

Tool Scores

• calculateB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states what the tool does (arithmetic operations) without mentioning any behavioral traits like error handling (e.g., division by zero), precision limits, input validation, or response format. This leaves significant gaps for a tool that performs calculations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function with zero wasted words. It is appropriately sized and front-loaded, making it easy for an agent to quickly understand the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (performing calculations with potential behavioral nuances) and the absence of both annotations and an output schema, the description is incomplete. It lacks information on error conditions, result formatting, or operational limits, which are crucial for an AI agent to use this tool correctly in varied contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description adds value by specifying the types of calculations supported (add, subtract, multiply, divide), which provides semantic context beyond the empty schema, though it doesn't detail parameter formats since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as performing basic arithmetic calculations with specific operations listed (add, subtract, multiply, divide). It uses a specific verb ('performs') and identifies the resource (calculations), though it doesn't distinguish from siblings since this is the only calculation tool among weather and greeting siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or context for choosing this tool over other methods, leaving the agent with no usage direction beyond the stated purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getWeatherAlertsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves alerts but does not mention critical details like rate limits, authentication needs, error handling, or what constitutes an 'active' alert. This leaves significant gaps in understanding the tool's operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (retrieving dynamic data like weather alerts) and the absence of annotations and an output schema, the description is insufficient. It does not explain return values, error conditions, or behavioral constraints, leaving the agent with incomplete information for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not add parameter details, and it implicitly clarifies that no inputs are required by specifying 'for a US state' without listing parameters, which aligns with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('active weather alerts for a US state'), making the tool's purpose immediately understandable. However, it does not explicitly differentiate from sibling tools like 'getWeatherForecast', which might offer related but distinct functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as 'getWeatherForecast' or 'calculate'. It lacks context about prerequisites, exclusions, or specific scenarios where this tool is preferred, leaving usage decisions ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getWeatherForecastB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on traits such as rate limits, authentication needs, error handling, or what the forecast includes (e.g., time range, metrics). This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function and input requirement without any wasted words. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (weather forecasting), lack of annotations, and no output schema, the description is minimally adequate. It covers the basic purpose and input but misses details like output format, error cases, or behavioral constraints, leaving room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds value by specifying that coordinates are required for location, which clarifies the input expectation beyond the empty schema, justifying a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('weather forecast'), and specifies the required input ('for a specific location using coordinates'). However, it doesn't differentiate from sibling tools like 'getWeatherAlerts' or 'calculate', which would require explicit comparison to earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'getWeatherAlerts' or 'calculate'. It mentions the input requirement (coordinates) but offers no context about appropriate use cases, exclusions, or prerequisites, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sayHelloB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'says hello', which implies a read-only or output action, but doesn't clarify if it requires any permissions, has side effects, or details the response format. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality ('Says hello to a person by name') with zero wasted words. It is appropriately sized for a simple tool and earns its place by clearly stating the purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks details on behavioral traits, usage context, or output. For such a simple tool, this is acceptable but leaves clear gaps, making it a baseline viable description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description adds value by implying the tool might operate on a person's name, but since there are no parameters, it doesn't need to compensate for schema gaps. A baseline of 4 is appropriate as the description provides some semantic context without parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('says') and resource ('hello to a person by name'), making the purpose immediately understandable. However, it doesn't distinguish this tool from potential sibling tools (like 'calculate' or weather-related tools), which would require a 5. The description avoids being vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as the sibling tools 'calculate', 'getWeatherAlerts', or 'getWeatherForecast'. It implies usage for greeting purposes but lacks explicit context, exclusions, or comparisons, leaving the agent without direction on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.