Server Quality Checklist

Tool Scores

• open_folderA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states that the tool opens a folder in the file manager, but does not mention what happens if the path does not exist, whether it creates missing folders, or if any output is returned. This is insufficient for a tool with zero annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with an example, front-loaded with the core purpose. There is no wasted text, and the example is directly relevant. This is exemplary conciseness for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description is minimally sufficient. However, it lacks context about edge cases (e.g., invalid paths, cross-platform behavior) and whether it returns a success indicator. Given the absence of annotations, a bit more detail would improve completeness, but it's not drastically inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for the 'path' parameter (0% coverage), so the description must compensate. The example 'path='C:/Users/xxx/Desktop'' provides a concrete format, but it does not explicitly explain the parameter's meaning or any constraints (e.g., absolute vs. relative paths). The parameter name is self-explanatory but the description adds only marginal value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Open folder in system file manager.' It uses a specific verb (open) and resource (folder), and the distinction from siblings like find_file (searching) and run_cli (command execution) is evident from the phrasing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool's usage is implied by the description and the provided example, but there is no explicit guidance on when to use it over alternatives or any exclusions. For instance, it doesn't mention that find_file might be more appropriate for locating files or that run_cli is for command-line operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• confirmA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: waiting (blocking) and timeout defaulting to reject, which is crucial for a confirmation tool. However, it does not state what the tool returns on success or whether it returns a boolean or other result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus an example. It is front-loaded with the primary action, includes a relevant example, and contains no extraneous information. Every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers the essential timeout behavior but omits the return value/result semantics. It also doesn't mention any side effects or environment requirements. While not severely lacking, the missing return info leaves a gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single prompt parameter with no description (0% coverage). The description provides an example (confirm(prompt='确认执行该操作？')) that illustrates the parameter's usage as a question string, but it does not elaborate on expected format or content beyond the example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool initiates and waits for a voice security confirmation, which is a specific verb+resource combination. It is distinctly different from sibling tools like run_cli, find_file, open_folder, and speak. The timeout behavior adds further specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for security-related confirmations via the term '语音安全确认' (voice security confirmation) and provides an example, but it does not explicitly state when to use this tool versus alternatives or any prerequisites. It gives context but lacks clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_fileA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the core behavior (fuzzy match by filename) and provides an example with pattern and directory, but it does not explain return values, scope of search (e.g., recursive), or any limitations. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence with a practical example. It immediately states the purpose and includes a concrete usage illustration, making it both concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter search tool, the description and example cover the essential usage. However, the lack of any output schema means the agent does not know what to expect as a return, and the description does not mention that (e.g., list of paths, matches). Given low schema coverage and no annotations, slightly more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the schema only provides names and a default for directory. The example partially compensates by showing pattern='报告' and directory='C:/Users/xxx/Desktop', implying that pattern is a filename keyword and directory is the folder to search. However, it does not fully explain the parameters' semantics beyond the example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: fuzzy search files by filename. It uses a specific verb (search) and resource (files), and distinguishes it from siblings like run_cli or open_folder which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (searching for files by name) through the example, and the sibling tools are not similar, so there is no confusion. However, it does not explicitly state when not to use it or mention alternatives, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_cliA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It clearly explains the safety triage: whitelisted commands execute, dangerous ones are rejected, and others need voice confirmation. This is valuable, though it does not mention return format or potential side effects, leaving room for improvement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary verb and resource, then adds safety behavior and an example. Every part is relevant and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (shell execution with security controls) and absence of output schema/annotations, the description covers the core function, safety policy, and an example. Missing details like return value or cwd behavior are notable but do not undermine the overall utility; the description is sufficiently complete for typical agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description compensates by providing a usage example ('例句：run_cli(command='dir')') that clarifies the 'command' parameter. However, the 'cwd' parameter receives no explanation, so the description only partially compensates for the schema's lack of semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes shell commands ('执行 shell 命令'), using a specific verb and resource. It also distinguishes itself from siblings by describing its safety policy (whitelist, rejection, confirmation), which is unique among the listed tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implicit usage guidance by explaining when commands run directly, when they are rejected, and when confirmation is needed. While it doesn't explicitly name alternatives, the inclusion of a whitelist and confirmation workflow gives practical context for when to invoke this tool versus others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speakA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It states that it broadcasts text via TTS, but does not mention any side effects, return values, or whether it blocks. For a simple TTS tool, this is minimal but adequate, though more detail (e.g., no return value) would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: one sentence stating purpose and one example. It is front-loaded with the core action and usage, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one string parameter, no output schema, no annotations), the description adequately covers purpose and usage. It could mention whether the tool returns anything or if it is synchronous, but for a TTS utility this is not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions, and schema coverage is 0%. The description compensates with a concrete example: speak(text='已打开文件夹'), which demonstrates both the parameter name and expected format. This adds meaning beyond the schema's bare type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool does TTS (text-to-speech) for reading back results. The verb '播报' (broadcast) is specific and the resource is text. It distinguishes from siblings by its unique purpose of speaking text aloud.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase '供工具回读结果' indicates when to use: when the tool needs to read back results to the user. It does not explicitly mention alternatives, but the purpose is clear enough for an agent to infer appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.