Server Quality Checklist

Tool Scores

• exec_commandB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It discloses that write operations require user confirmation via the 'confirmed' parameter, adding safety context. However, it does not detail permissions, logging, or implications of arbitrary command execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with bulleted parameter list. It front-loads the main action. Could be slightly more structured but efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description lacks usage context, differentiation from sibling tools, and behavioral details. An agent would struggle to know when to choose this tool over alternatives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description adds meaning to all three parameters: server (Chinese name), command (shell command), confirmed (confirmation for write ops, default false). This compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes arbitrary shell commands on a specified server. It distinguishes from siblings by its focus on single-server command execution, though not explicitly mentioned.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings like batch_exec or read_logs. The description lacks explicit when/when-not scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transfer_fileA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions that uploads require confirmation as a write operation, but omits other behaviors such as overwrite policy, authentication requirements, size limits, or error handling. This is insufficient for a complete behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a bulleted list of parameters. It is front-loaded with the core purpose and parameter list. It wastes no words, though it could be slightly better structured with headings.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters and an output schema (not shown but present). The description covers all parameters but lacks prerequisites, error cases, or behavioral notes beyond the confirmation requirement. It is adequate for simple usage but incomplete for complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It explains all five parameters: server (Chinese name), direction (upload/download), local and remote paths, and confirmed (write confirmation). This adds critical meaning not present in the schema, though it could be more precise (e.g., expected formats).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states '在本地和服务器之间传输文件（SFTP）' which translates to 'Transfer files between local and server (SFTP).' This specifies a specific verb, resource, and protocol, and it distinguishes the tool from siblings like exec_command or list_servers that are not for file transfer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides parameter details but no explicit guidance on when to use this tool versus alternatives. It implicitly indicates use for file transfers via SFTP, but does not mention exclusions or conditions like when not to use it or compare with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_serversA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the basic listing behavior. It fails to disclose potential traits like access restrictions, rate limits, or whether the list includes all servers without filtering or pagination.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and includes key output details, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an existing output schema, the description adequately covers the tool's action and output details (name, IP, tags, project). No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description cannot add value beyond the schema. Baseline for 0 parameters is 4, and the description correctly omits unnecessary parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and resource 'servers', and specifies included fields (name, IP, tags, project). It distinguishes well from sibling tools that focus on executing commands, reading logs, or managing services.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving server information, but provides no explicit when-to-use or when-not-to-use guidance, nor alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_logsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It describes the tool as a read operation and explains parameters but does not disclose potential side effects, authentication requirements, or rate limits. The nature of reading logs is low risk so this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient and front-loaded with the purpose. It uses a bullet-style layout. However, it repeats parameter names in the same fashion as the schema, slightly increasing redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and the moderate complexity (5 parameters, 1 required), the description covers parameter usage well. It lacks details about output format, but this is partially offset by the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description compensates thoroughly by explaining each parameter with examples and default values. It adds meaning beyond the schema structure, especially for server, project, log_type, and keyword.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads log files from a server and project, specifying the action and resource. It distinguishes from sibling tools like exec_command or list_servers which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides detailed parameter guidance, including defaults and when to omit optional fields. However, it does not explicitly compare with sibling tools or state when not to use this tool, leaving usage context somewhat implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• service_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses basic behavior (checking status and resources) and default services, but lacks information on side effects, permissions, or rate limits. It is adequate but not detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with two sentences plus parameter explanations. Every sentence adds value, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity, the description covers purpose and parameters well. Output schema exists to explain return values, so no need for that here. It misses only minor details like error handling or prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates by explaining 'server' is the Chinese name and 'services' is an optional list with defaults (nginx, php-fpm, mysql, redis, system resources). This adds valuable context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks running status and system resources of key services on a server, with explicit verb and resource. It distinguishes well from sibling tools like exec_command and read_logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case (checking service status) but does not explicitly state when to use this tool versus alternatives. No exclusion criteria or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_execA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It mentions multi-server execution and the confirmed flag for write operations, but lacks details on permissions, error handling, or limits. Some transparency but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with a clear main line followed by a bullet list for parameters and a constraint. No unnecessary words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters (1 required) and an output schema existing, the description covers the essential usage and constraints. It could mention what the tool returns (though output schema may handle that), but it's adequate for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must add meaning. It fully explains each parameter: command is a Shell command, tag filters servers, servers is a list of server names, confirmed is for write confirmation. This goes beyond the schema's type-only definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'execute the same command on multiple servers simultaneously', which is a specific verb+resource. It distinguishes from sibling tools like exec_command (likely single server) by emphasizing multi-server execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides guidelines: parameters tag and servers must have at least one, and for write operations, confirmed must be True. It doesn't explicitly mention when to use alternatives like exec_command, but the constraint is helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.