Server Quality Checklist

Tool Scores

• biome_lintC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description simply restates the non-modifying behavior. It adds no additional behavioral context such as output format, rate limits, or side effects beyond what annotations already provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, scannable sentence with zero wasted words. It efficiently communicates the primary action and the read-only safety guarantee.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of sibling tools and three optional parameters, the description is too sparse. It does not explain how to scope the lint run, what the output contains, or when to prefer this tool over biome_check. The output schema and annotations partially compensate, but important gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the tool description does not mention the level, paths, or changed_only parameters. An agent must infer meaning from parameter names and defaults, which is insufficient for a three-parameter tool with no other explanatory text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as running Biome lint diagnostics and explicitly states it does not modify files, making the core purpose obvious. However, it does not explicitly distinguish itself from sibling tools like biome_check or biome_format, so it lacks explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus biome_check or biome_format. The read-only nature is implied, but there are no prerequisites, exclusions, or contextual clues to help an agent choose this tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_cacheC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only says 'Refresh' without explaining side effects, whether the operation is destructive, permission requirements, or the effect of the 'force' parameter. The lack of detail leaves the agent guessing about runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the main purpose. However, it is so terse that it borders on under-specification, lacking any supporting details that would make it truly helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 optional parameter), the description still leaves major gaps: no explanation of when to refresh, what 'force' does, what the operation modifies, or any side effects. The presence of an output schema mitigates return-value ambiguity, but the description is too sparse to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the 'force' parameter at all. The parameter name gives a hint ('force' implies overriding something), but the description adds no meaning beyond the schema, which itself provides no description. The tool fails to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Refresh') and the resource ('cached Biome documentation'). It distinguishes itself from sibling tools that perform code checks, linting, formatting, or search, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided for when to use this tool versus alternatives. There is no mention of prerequisites, recommended frequency, or scenarios where the cache should be refreshed. The usage context is entirely implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• biome_formatB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds 'without modifying files', reinforcing the read-only nature but without additional behavioral detail. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One short sentence, front-loaded, with no redundant wording. Efficient but could carry more information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 2 parameters and no schema description coverage, the description should explain parameter usage and differentiate from sibling tools. Output schema and annotations mitigate some gaps, but the description remains insufficient for full usability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description provides no explanation for 'paths' or 'changed_only' parameters, leaving the agent to infer meaning solely from names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb 'Check' with resource 'Biome formatting' and specifies 'without modifying files', distinguishing it from mutation tools and aligning with its name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use biome_format vs sibling tools like biome_check or biome_lint. The description doesn't mention alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentationB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses that the search is against 'cached' documentation, implying a local, read-only operation, but it does not elaborate on staleness, caching behavior, or any side effects. The word 'cached' adds some transparency, but the description remains minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no unnecessary information. Every word earns its place, and the structure is immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, a short description might be adequate. However, the lack of query-format guidance and any usage context leaves gaps. It is adequate but could benefit from a note about what the query should contain or that the search is against a local cache.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description contains no additional meaning about the 'query' parameter. It does not clarify what format the query should take (e.g., natural language, keyword, regex) or how to construct effective searches. The description fails to compensate for the schema's lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Search' and identifies the resource as 'cached Biome documentation', clearly distinguishing it from sibling tools like biome_check, biome_lint, and biome_format. It leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (searching documentation) but provides no explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives. Since the tool's purpose is self-evident from its name and description, it earns an implied-usage score rather than a lower one.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• biome_checkA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, while the description adds the specific scope (formatting, linting, assist) and confirms no files are written. This aligns with the annotations and provides additional behavioral context without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the core action and a key constraint. Every word earns its place, with no redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the primary function and non-destructive behavior, and an output schema exists to explain return values. However, the absence of any parameter semantics and explicit guidance relative to sibling tools leaves the overall context incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no explanation of the 'paths' and 'changed_only' parameters. The agent must infer their meaning solely from the parameter names, which is insufficient for reliable invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Run') and resource ('Biome formatting, linting, and assist checks'), and explicitly states it does not write files. This clearly distinguishes it from sibling tools like biome_format and biome_lint, which focus on individual operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is a combined check covering multiple Biome operations, but it does not explicitly state when to prefer biome_check over biome_format or biome_lint, nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.