Server Quality Checklist

Tool Scores

• disconnectA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the action but does not disclose any side effects, preconditions (e.g., must be connected), or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded and contains no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description adequately conveys the tool's function. However, it leaves out any mention of return values or preconditions, which could be useful for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the description provides no parameter information. Per baseline, 0 parameters warrant a score of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action (断开, disconnect) and the resource (交换机的Console口, switch's console port). It is distinguishable from the sibling tool 'connect'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the tool name and description (disconnect from a switch's console port), but there is no explicit guidance on when to use it or how it relates to alternatives like 'connect'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_commandA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It goes beyond the schema by explaining that newlines are auto-added, that timeout should be increased for long outputs, and that the prompt changes when entering system-view. These are useful behavioral details. However, it does not disclose the exact return format or potential error behaviors, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear purpose statement, followed by a well-organized list of common commands and a note about system-view. While it is longer than a typical description due to the examples, every line contributes useful information and there is no fluff or repetition. It is structured effectively for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only 2 parameters and no output schema or annotations, the description is fairly comprehensive. It explains the core action, provides usage examples, and gives practical tips like timeout adjustment and prompt change behavior. It does not specify the return format or error handling, but for a simple command-sending tool, the level of detail is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters (command and timeout) fully, so the baseline is 3. The description adds value by providing example command syntax and clarifying that newlines are automatically added, plus a recommendation to increase timeout for long outputs. This enhances the semantic understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: sending a command to the connected switch and retrieving the result (向已连接的交换机发送一条命令，并获取返回结果). It includes numerous command examples, making the purpose concrete. However, it does not explicitly distinguish itself from the sibling tool send_config, which likely also sends commands, so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides many example commands (display version, ping, system-view) that illustrate typical usage, and notes about auto-added newline and timeout for long outputs. However, it does not explicitly state when to use this tool versus alternatives like send_config or get_status, nor does it mention any exclusions. The usage context is implied through examples rather than explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It lists the output fields (connected, port, baud rate, device type), which is useful, but it does not explicitly state that the tool is read-only, what happens when not connected, or whether it may block or throw errors. The verb '获取' implies a read operation, but the description could be more transparent about side effects and edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately conveys the purpose and key output details. It contains no fluff or repetition, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description is fairly complete: it states the purpose and enumerates the expected return fields. It falls short of describing error behavior or the exact response format, but for a status-checking tool, it covers the core information well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema coverage is trivially 100%. According to the rubric, the baseline is 4. The description adds no parameter information because there are none to describe; this is appropriate and does not detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb '获取' (get) and the specific resource '当前连接状态' (current connection status), and enumerates the included details (connected, port, baud rate, device type). This is a specific, non-tautological purpose that effectively distinguishes the tool from siblings like connect, disconnect, and list_ports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool over alternatives or provide exclusion criteria. However, the unique purpose of checking connection status is implied, which gives some guidance. There is no mention of 'use this when you need to verify connection' or 'use list_ports to enumerate available ports instead.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_configA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that commands are executed sequentially: '自动逐条执行' (automatically executes one by one) and '按顺序执行' (executed in order), and notes the system-view prerequisite. However, it does not mention error handling behavior, whether execution stops on failure, or any side effects beyond configuration changes, which are gaps given the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main purpose, followed by a prerequisite and useful example commands. It is reasonably concise, though the example list is somewhat long. Each part serves a purpose: examples clarify command format and context. No unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and no output schema or annotations, the description provides the essential usage context: batch execution, order, and prerequisite. However, it omits important operational details such as whether an active connection is required and what happens when a command fails. Since annotations and output schema are absent, the description should have covered these to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for the 'commands' parameter (list of commands, one per line). The tool description adds value beyond the schema by providing example commands, clarifying the order of execution, and emphasizing the need for system-view context. This enriches the understanding of what valid input looks like.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: '批量发送配置命令给交换机' (batch send configuration commands to switch), with '自动逐条执行' (automatically executes one by one). It also emphasizes '适合批量配置场景' (suitable for batch configuration scenarios), which distinguishes it from sibling tools like send_command that target single commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: '适合批量配置场景' (suitable for batch configuration scenarios). It also specifies a prerequisite: '使用前请确保已进入 system-view（系统视图）' (before use, ensure you have entered system-view). However, it does not explicitly mention alternatives or when not to use it, so it falls short of a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connectA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions common baud rates and port examples, which adds operational context, but it does not explain what happens on connection failure, whether the connection is persistent, or how to handle a busy port. These gaps are notable for a connection-establishing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it opens with the tool's purpose, then provides relevant practical details (baud rates and port examples) in a list-like format. No sentence is wasted, and the information is front-loaded for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has five parameters and no output schema, the description provides a reasonable level of completeness by explaining the connection action and giving examples for common parameters. It does not mention return values or session management, but those may be implied by the sibling tools. The lack of annotations is partially compensated by the practical guidance included.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers parameter descriptions fully, but the description adds value by providing OS-specific port examples (Windows, Linux, macOS) and noting that some devices use 115200 baud. This goes beyond the schema, which only gives a generic example, and helps the agent select appropriate parameter values in different environments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (connecting via Console port) and the target resource (network devices such as Huawei/H3C/Cisco switches). It distinguishes itself from sibling tools by focusing solely on establishing the connection, unlike send_command or send_config which operate after a connection exists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by mentioning that after connecting, commands can be sent, which implies this tool is a prerequisite for send_command/send_config. It also offers practical OS-specific port examples and common baud rates. However, it does not explicitly state when not to use this tool or mention alternatives like list_ports for discovering available ports.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_portsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It implies a read-only operation by stating it lists available ports, but it does not disclose the output format, whether it modifies anything, or any system prerequisites. For a simple listing tool, this is adequate but not rich. There is no contradiction with annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary action ('列出所有可用的串口'), followed by a concise usage note. Every sentence earns its place, with no superfluous content. It is compact and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description is adequately complete. It states what the tool does and when to use it, which is sufficient for an agent to select and invoke the tool. It could mention the return format or that it is read-only explicitly, but those are minor omissions for a trivial list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters in the input schema, and the schema description coverage is 100%. The description adds no parameter-specific information because there are no parameters to document. Per the rubric, 0 parameters corresponds to a baseline score of 4, and the description does not need to compensate for any parameter documentation gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: '列出计算机上所有可用的串口（COM口）' (List all available serial ports on the computer). It uses a specific verb ('列出' / list) and resource ('串口' / serial ports), and further clarifies its purpose for finding the port of a console-connected switch. This distinguishes it from the sibling tools like connect, send_command, etc., which handle other aspects of serial communication.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it is used to find the port number where a switch is connected via Console cable ('用于查找通过Console线连接的交换机所在的端口号'). Although it does not explicitly mention when not to use this tool or name alternative tools, the context is sufficient for an agent to know it is a preliminary discovery step before connect or send_config. The absence of exclusions is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.