Server Quality Checklist

Tool Scores

• show_icon_infoC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only states a generic action. It does not mention whether the operation is read-only, whether network calls or permissions are needed, or any side effects. The verb 'show' implies read-only but this is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded and free of redundancy. However, it is under-specified, which slightly detracts from what would otherwise be a very concise entry.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two parameters, no annotations, and a workflow implied by siblings, the description is incomplete. It does not explain the relationship to find_icon, how query affects results, or what 'detailed information' includes. The output schema exists but does not cover usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate. It fails to explain the purpose of 'query' or 'count', leaving the agent without information on how to fill these parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Show detailed information about found icons' clearly specifies a verb (show) and resource (detailed information about found icons). It is specific enough to distinguish from siblings like download_icons or roll_dice, though it does not explicitly contrast with find_icon.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit usage guidance is provided. The phrase 'found icons' vaguely implies a prior step, but there is no mention of when to use this tool versus find_icon or other siblings, nor any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_iconC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It merely states the action without detailing what the function returns, whether it can return multiple results, whether it accesses external resources, or any side effects. This is insufficient for an agent to predict tool behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the main action. It wastes no words and is easy to parse, though its brevity comes at the cost of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and minimal parameter descriptions, the overall context is incomplete. The agent has no information about return format, pagination, icon sources, or interpretation of results. This is inadequate for reliable tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'given query' which maps to the required 'query' parameter, but it does not explain the 'count' parameter at all. With 0% schema description coverage, the description needed to clarify all parameters, but it only covers half of them and even then vaguely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Find') and resource ('icon'), and the query parameter is mentioned. It identifies the tool's core function, though it does not distinguish it from siblings like download_icons or show_icon_info, which could also involve finding icons.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance is provided. The description does not indicate when to use find_icon versus alternatives like download_icons or web_search, nor does it specify any prerequisites or exclusions. The intended context is implied only by the tool's name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• roll_diceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It simply states the action without revealing whether the operation is random, side-effect-free, or requires permissions. The agent has no information about error handling or validation of the notation format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one short sentence, no wasted words). It is appropriately front-loaded with the core action. While under-specification is an issue, that is penalized in other dimensions; for structure and efficiency, this is near-optimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and a simple tool, the description is incomplete for correct invocation. It omits parameter semantics and fails to provide enough context about how the notation string should be formatted. An agent would likely need to guess or rely on the output schema for clues, making this insufficient for a 2-parameter tool with 0% schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description fails to compensate. It mentions 'notation' but does not explain what constitutes valid dice notation (e.g., '2d6'), nor does the description mention the 'num_rolls' parameter at all. The agent is left without any semantic guidance for the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('roll') and resource ('dice'), and clearly distinguishes this tool from its unrelated siblings (web_search, find_icon, etc.). The term 'notation' hints at dice notation, making the purpose unambiguous for a dice-rolling tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. While siblings are unrelated, the description does not mention use cases, prerequisites, or exclusions. An agent must rely solely on the tool name and vague description to decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for disclosing behavior. It only states that the tool searches the web, offering no information on read-only nature, result format, latency, or potential side effects. This is insufficient for an agent to anticipate outcomes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence: 'Search the web for information about the given query.' It is front-loaded with the action verb, contains no unnecessary words, and is easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's simplicity, the description is incomplete. It does not mention the return value or output structure, and there is no usage guidance or behavioral context. The presence of an output schema does not mitigate the absence of any description of what the tool returns or how results are presented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single parameter 'query' with no description, and the schema description coverage is 0%. The description says 'information about the given query,' but this merely restates the parameter name without adding format, domain, or example guidance, thus failing to compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Search' with a clear resource 'the web' and specifies the action's purpose: 'information about the given query.' This unambiguously distinguishes it from sibling tools like 'roll_dice' or 'find_icon', which operate on unrelated domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description merely states the action, leaving the agent without context on appropriate invocation, exclusions, or preferred use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_iconsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states that icons are downloaded to a local directory, but does not clarify important behaviors such as whether files are overwritten, whether a directory is created if missing, what file naming conventions are used, or what the tool returns upon success or failure. For a write side-effect tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no redundant words, making it concise and front-loaded. It could be slightly more informative without becoming bloated, but it does not waste words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description should explain the full context of the tool's behavior, including return values, error conditions, and parameter effects. The provided one-liner does not cover these, leaving significant gaps for the agent to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for explaining parameters. It only implicitly references 'query' via the phrase 'given query', but leaves 'size' and 'count' completely undefined. The tool does not provide any guidance on how these affect the download, forcing the agent to guess.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Download icons for the given query to local directory'. It specifies the action (download), the object (icons), the source (query), and the destination (local directory). This distinguishes it from sibling tools like find_icon and show_icon_info, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you need to download icons based on a query to a local directory. However, it does not explicitly mention when not to use it or name alternative tools like find_icon for searching, so it falls short of the highest score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.