Server Quality Checklist

Tool Scores

• roll_diceC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for disclosing behavior. It only states that dice are rolled, without mentioning randomness, error handling for invalid notation, return format, or how num_rolls affects output. The presence of an output schema is not leveraged in the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of one short, direct sentence with no redundant wording, achieving excellent conciseness. However, the brevity sacrifices essential details, making it slightly less 'appropriately sized' for the tool's two-parameter surface.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters, no annotations, and an output schema, the description should clarify the expected notation syntax and what the tool returns (e.g., individual rolls, aggregate totals). It does neither, leaving basic usage ambiguous for an agent that needs to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description refers only to 'given notation' and does not explain its expected format (e.g., dice notation) or the role of num_rolls. Schema description coverage is 0%, so the parameter meanings must be guessed solely from their names, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action (roll dice) and the key input (notation), making it distinct from unrelated sibling tools like web_search or get_quote. However, it does not explicitly clarify that 'notation' follows dice notation (e.g., '2d6'), leaving some ambiguity for an agent unfamiliar with RPG conventions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool or when to prefer alternatives. The sibling tools are unrelated, so there is no explicit contrast or scenario described, leaving the agent to infer the appropriate context solely from the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only mentions the basic action without covering expected output, limitations, rate limits, or other behavioral traits, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero wasted words. It earns every word, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is very simple with one parameter and an output schema, so a minimal description is somewhat adequate. However, it lacks usage guidance and behavioral disclosures, leaving the description incomplete even for a low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The phrase 'given query' adds minimal meaning beyond the parameter name 'query' and type 'string', offering almost no extra semantic value or formatting guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Search') and resource ('the web'), making the tool's purpose unambiguous. It does not explicitly distinguish from sibling tools, but siblings are clearly different domains (dice, text, quotes), so the purpose is effectively clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool or when to prefer an alternative. The description merely states what it does, leaving the agent to infer usage context without any exclusions or scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quoteB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It mentions the external API but does not state that the operation is read-only, that it requires network access, or what happens on failure (e.g., empty result or error). This is minimal disclosure beyond the core action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short but contains redundancy: the first sentence already states it gets a random dad joke, and the second sentence repeats the same information with a 'Returns:' label. Since an output schema exists, this second sentence adds no value and could be removed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless tool with an output schema, the essential behavior is covered. However, the description omits potential operational context such as network dependency, error handling, or rate limits, and the tool name 'get_quote' may mislead users expecting quotes instead of dad jokes. This leaves room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is an empty object with 100% coverage. Per the rubric, a baseline of 4 applies to parameterless tools, and the description correctly does not need to elaborate on parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches a random dad joke from a specific API (icanhazdadjoke), using the verb 'Get' with a concrete resource. This is distinct from sibling tools such as web_search and roll_dice, leaving no ambiguity about its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no information about when to use this tool compared to alternatives. It does not mention exclusions, prerequisites, or contexts where it would be preferred over roll_dice or web_search, offering no usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• text_utilsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the burden of disclosing behavior. It thoroughly describes each operation's effect (e.g., 'uppercase: Convert text to uppercase') and notes the random_case behavior. However, it does not mention edge cases such as empty input or invalid operation handling, nor does it describe the return structure, though an output schema exists. This is adequate for a pure function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a one-line intro followed by a bulleted list of operations. Every line conveys necessary information, and the format is front-loaded with the primary purpose. No wasteful words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has only two simple parameters and an output schema, the description is complete. It covers all operation options and their semantics, making it sufficient for an agent to select and invoke the tool. The lack of detailed error handling or whitespace notes is a minor omission for such a deterministic utility.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only parameter names and types with zero descriptions. The description compensates fully by explaining the 'operation' parameter enumerations and their meanings, and by defining 'text' as the 'input text.' This gives the agent everything needed to construct valid arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Perform various text operations on the input text,' a specific verb+resource statement that immediately defines the tool's scope. It enumerates nine distinct operations, distinguishing it from sibling tools like web_search and roll_dice, which are unrelated. The tool name 'text_utils' is vague, but the description fully disambiguates it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly frames the tool's use case as text transformation and lists all supported operations, giving an agent precise criteria for selection. It does not explicitly state when NOT to use this tool, but the sibling tools are so different that no exclusions are necessary. The context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.