Server Quality Checklist

Tool Scores

• timerC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions that 'You don't need to say anything else after answering with this tool', which hints at a silent or automated post-invocation behavior, but doesn't clarify key aspects like whether the timer runs in the background, how it notifies upon completion, or if it's interruptible. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two short sentences that are front-loaded: the first states the core purpose, and the second provides a usage note. There is no wasted text, and every sentence serves a clear purpose, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no annotations), the description is incomplete. It lacks details on behavioral traits (e.g., how the timer operates or notifies), doesn't explain the return values or any post-invocation effects beyond a vague hint, and fails to address potential complexities like error handling or duration limits, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'duration' clearly documented in the schema as 'The duration of the timer in seconds'. The description doesn't add any additional meaning or context beyond what the schema provides, such as acceptable ranges or units, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start a timer') and identifies the resource (timer), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'stopwatch' or 'conversion', which could have overlapping timing functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance with 'You don't need to say anything else after answering with this tool', which suggests a post-invocation behavior but doesn't offer when to use this tool versus alternatives like 'stopwatch'. No explicit when/when-not scenarios or sibling tool comparisons are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stopwatchB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool starts a stopwatch and that no further action is needed, but it doesn't disclose key behavioral traits such as whether the stopwatch runs in the background, how it's displayed or accessed, if it has rate limits, or what happens on subsequent invocations. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence clearly states the purpose, and the second adds usage context. Both sentences earn their place by providing essential information without waste. However, it could be slightly more structured by explicitly mentioning the lack of parameters or sibling differentiation, keeping it from a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a tool that starts a stopwatch) and the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a confirmation, an ID, or nothing), how the stopwatch is managed, or any error conditions. For a tool with no structured data to rely on, this leaves the agent with insufficient information to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add any parameter information, which is appropriate. A baseline score of 4 is given because the schema fully covers the parameters (none exist), and the description doesn't need to compensate, but it doesn't explicitly note the lack of parameters, preventing a perfect score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Start a stopwatch that counts up from zero.' It specifies the verb ('Start') and resource ('stopwatch'), and distinguishes it from sibling tools like 'timer' by focusing on counting up from zero rather than counting down. However, it doesn't explicitly differentiate from all siblings (e.g., 'conversion', 'display-fact'), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage guidance: 'You don't need to say anything else after answering with this tool.' This implies a specific usage context (e.g., after starting the stopwatch, no further action is needed), but it doesn't explain when to use this tool versus alternatives like 'timer' or other siblings. No explicit when/when-not scenarios or alternatives are mentioned, leaving gaps in guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• conversionA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'display a widget' and 'real-time conversion,' but does not specify whether this is a read-only or interactive operation, what happens on invocation (e.g., UI display, data processing), or any constraints like rate limits or permissions. The description adds minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences: one stating the purpose and one providing an example. It is front-loaded with the core functionality, though the example is detailed and could be slightly verbose. Overall, it avoids unnecessary information and earns its place efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters with nested objects, no output schema, and no annotations), the description is moderately complete. It covers the purpose and provides an example for parameters, but lacks details on behavioral traits, return values, or error handling. This leaves gaps for an agent to fully understand tool behavior without annotations or output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the parameters 'units' and 'initialValue' thoroughly. The description adds value by providing an example input that illustrates how to structure the parameters, including nested objects and formula placeholders, which clarifies semantics beyond the schema's technical definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Display a unit conversion widget that allows real-time conversion between multiple units.' It specifies the verb ('display'), resource ('widget'), and functionality ('real-time conversion'), distinguishing it from sibling tools like display-fact, stopwatch, and timer which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for unit conversion scenarios but does not explicitly state when to use this tool versus alternatives. It provides an example input, which hints at context, but lacks guidance on prerequisites, exclusions, or comparisons to sibling tools. This leaves the agent to infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• display-factA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it displays facts, requires no additional output from the agent, and handles short factual answers. However, it doesn't mention potential limitations like character limits or formatting constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and front-loaded: the first sentence states the purpose, the second provides usage guidelines, and the third gives a concrete example. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, no annotations), the description is nearly complete. It covers purpose, usage, and provides an example. The only minor gap is the lack of explicit mention about what happens after tool invocation (though 'You don't need to say anything else' implies the tool handles output).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value beyond the schema by providing an example that illustrates parameter usage, but doesn't explain semantics beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Display a simple fact') and resource ('fact'), and distinguishes it from siblings by specifying its use case for short factual answers. It provides a concrete example that reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('when answering a users question that has a short factual answer') and provides clear exclusion guidance ('You don't need to say anything else after answering with this tool'). This differentiates it from other tools like conversion, stopwatch, and timer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.