Server Quality Checklist

Tool Scores

• roll_diceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the basic action without disclosing behavioral traits like output format, error handling, or constraints (e.g., notation syntax limits). It mentions 'notation' but doesn't explain what that entails, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loading the core action. It's appropriately sized for a simple tool, making every word count without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, and no output schema, the description is incomplete. It doesn't explain what 'notation' means, how results are returned, or any error cases, making it inadequate for a tool with two parameters and unspecified behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but only vaguely references 'notation' without explaining its meaning or format. It doesn't address the 'num_rolls' parameter at all, failing to add meaningful semantics beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Roll') and resource ('dice'), specifying it uses 'given notation' which distinguishes it from simple dice rolling. However, it doesn't explicitly differentiate from sibling tools like generate_qr_code or scientific_calculator, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or in what context. It lacks any mention of prerequisites, exclusions, or comparisons with sibling tools, leaving usage entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions searching the web but doesn't disclose traits like rate limits, authentication needs, result format, or potential side effects (e.g., network usage). This is a significant gap for a tool that interacts with external resources.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loaded with the core action. It's appropriately sized for a simple tool, making it easy to parse without unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of web searching (external interaction, no annotations, no output schema, and 0% schema coverage), the description is incomplete. It lacks details on behavior, results, error handling, or constraints, making it inadequate for an agent to use the tool effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, and the description adds minimal meaning beyond the schema. It implies the 'query' parameter is used for web searches but doesn't explain its semantics, such as expected format, length constraints, or how it influences results. This fails to compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Search') and resource ('the web'), making it immediately understandable. However, it doesn't differentiate from sibling tools like 'generate_qr_code' or 'roll_dice', which are unrelated, so it doesn't need sibling differentiation but could be more specific about what kind of web search it performs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or in what context. It simply states what it does without mentioning prerequisites, limitations, or scenarios where other tools might be more appropriate, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_qr_codeB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the return format (Base64 encoded PNG image) but lacks details on performance, rate limits, error handling, or side effects. For a tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by Args and Returns sections. It's appropriately sized with no redundant information, though the parameter explanations could be slightly more concise (e.g., the error correction percentages are detailed but not strictly necessary).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 4 parameters, the description is moderately complete. It covers parameters well and specifies the return format, but lacks context on behavioral aspects like error conditions or usage limits, making it adequate but with clear gaps for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides detailed semantic explanations for all four parameters: 'data' as text/URL to encode, 'error_correction' with levels and percentages, 'border' with minimum value, and 'box_size' with default. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Generate a QR code for the given data,' which is a specific verb+resource combination. However, it doesn't differentiate from sibling tools (roll_dice, scientific_calculator, web_search) since they serve completely different domains, so sibling differentiation isn't relevant here.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or scenarios where this tool is preferred over other methods for generating QR codes or handling the data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scientific_calculatorA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does this well by specifying the calculator's capabilities (scientific functions, complex numbers), default behavior (trigonometric functions work with radians), and conversion utilities (degrees, radians). However, it doesn't mention error handling, precision limits, or performance characteristics that would be useful for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by categorized bullet points and examples. Every sentence earns its place by providing specific, actionable information. It could be slightly more concise by combining some bullet points, but overall it's efficiently organized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (mathematical evaluation with many functions), no annotations, no output schema, and low schema coverage, the description does an excellent job of providing context. It covers capabilities, syntax, defaults, and examples. The main gap is the lack of information about return values or error cases, which would be helpful since there's no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage with only one parameter 'expression' of type string. The description compensates fully by providing extensive semantic context: it explains what the expression parameter should contain (mathematical notation), lists all supported operations and functions, shows syntax examples, and even provides conversion utilities. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Evaluate mathematical expressions using a scientific calculator' with a specific verb ('evaluate') and resource ('mathematical expressions'), distinguishing it from sibling tools like generate_qr_code, roll_dice, and web_search which have completely different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the extensive list of supported operations and examples, suggesting this tool is for mathematical evaluation. However, it doesn't explicitly state when to use this versus alternatives or provide any exclusion criteria, leaving the context somewhat implied rather than clearly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.