Server Quality Checklist

Tool Scores

• add_numbersB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states what the tool does but doesn't disclose behavioral traits like error handling, performance characteristics, or what happens with non-numeric inputs. For a tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any unnecessary words. It's appropriately sized and front-loaded, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, no annotations), the description is adequate but minimal. It covers the basic purpose but lacks context about when to use it, behavioral details, or output expectations, leaving some gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters clearly documented in the schema. The description adds no additional meaning beyond what the schema provides (e.g., no details on number types, ranges, or special cases), so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Add') and resource ('two numbers together'), making the purpose immediately understandable. It doesn't distinguish from siblings since this is a simple arithmetic operation, but the purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or comparison with sibling tools like 'echo' or 'get_current_time', leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• echoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states the tool echoes input text, implying a simple read-only operation, but doesn't cover aspects like error handling, performance, or side effects. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—'Echo back the input text' directly conveys the core function without fluff. It's appropriately sized for a simple tool and front-loaded with essential information, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks context on usage, behavior, or output. Without annotations or output schema, more detail would be helpful, but it's not critically incomplete for such a simple operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'text' fully documented in the schema as 'Text to echo back'. The description adds no additional meaning beyond this, such as format constraints or examples. Baseline 3 is appropriate when the schema does all the heavy lifting for parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Echo back the input text' clearly states the tool's function with a specific verb ('echo back') and resource ('input text'). It distinguishes from siblings like 'add_numbers' or 'get_current_time' by focusing on text reflection rather than computation or data retrieval. However, it doesn't explicitly differentiate from hypothetical similar tools (e.g., 'repeat_text'), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention use cases, prerequisites, or comparisons with sibling tools like 'get_flight_status' for different data needs. The agent must infer usage from the purpose alone, which is insufficient for optimal tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_timeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Get' implies a read-only operation, it doesn't explicitly state this or provide any additional behavioral context such as whether it requires authentication, has rate limits, or what format the time is returned in. The description is minimal and lacks behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise at just 5 words, front-loading the essential information with zero wasted words. Every word earns its place in communicating the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter tool with no annotations and no output schema, the description provides the basic purpose but lacks important context. It doesn't specify the format of the returned time (e.g., ISO string, timestamp), timezone handling, or any behavioral constraints. While the tool is simple, the description leaves gaps that could affect proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't mention parameters since none exist, which is correct for this context. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Get') and resource ('current date and time'), making the purpose immediately understandable. However, it doesn't distinguish this tool from potential siblings that might also retrieve time information, though none exist in the provided sibling list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While the sibling tools (add_numbers, echo, get_flight_status) are clearly different in function, the description doesn't explicitly state this or provide any context about appropriate use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_statusB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves flight status but omits critical details like whether it requires authentication, has rate limits, returns real-time or cached data, or handles errors. This leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action and resource, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not explain what the tool returns (e.g., status details like departure time, delays), error conditions, or operational constraints. For a tool with two parameters and no structured output, more context is needed to ensure effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear documentation for both parameters (flightNumber and airlineIata). The description adds minimal value beyond the schema, as it only mentions 'flight number' without elaborating on parameter usage or interactions. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get the latest status') and resource ('a flight'), using the flight number as the key identifier. It distinguishes this tool from its siblings (add_numbers, echo, get_current_time) by focusing on flight status retrieval rather than mathematical operations, text echoing, or time queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as other flight-related tools that might exist in a broader context (though none are listed among siblings here). It lacks any mention of prerequisites, constraints, or typical use cases beyond the basic function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.