Server Quality Checklist

Tool Scores

• dockerC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations exist, the description should bear the burden of behavioral disclosure. It only lists action names without indicating that start/stop/restart are mutating operations or that logs/stats are read-only, nor does it mention potential side effects, permissions, or state changes. The description adds minimal value beyond what the schema's enum already provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that efficiently lists core actions. However, it omits 'restart' and 'images' from the schema, which could mislead the agent into thinking those actions are unsupported. Despite this, it remains appropriately brief and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-action tool with 3 parameters and no output schema, the description is under-specified. It does not explain return values, error conditions, or when each action is appropriate. The schema covers parameter details, but the overall context for invoking this tool safely and effectively is incomplete, especially given the mutating actions present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all three parameters with 100% coverage, including descriptions for action, container, and log_lines. The description merely repeats a subset of the action enum values and does not add extra meaning about parameter relationships, such as container being required for certain actions, which the schema already states. This meets the baseline but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it manages Docker containers and lists concrete actions (list, start, stop, logs, stats), making the purpose understandable. It is distinct from sibling tools like run_command or manage_service because of the Docker-specific resource, though the verb 'manage' is generic and does not fully enumerate all schema actions (e.g., restart, images).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as manage_service or run_command. The description does not specify prerequisites (e.g., Docker daemon) or exclusion criteria, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• firewallC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It lists actions like 'delete' and even 'reset' (the latter only in schema) which are destructive, but does not warn about potential side effects, permission requirements, or reversibility. The description gives no safety-relevant context beyond naming the actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with the core purpose front-loaded. Every word contributes value, and it avoids redundancy with the schema. It is appropriately sized for a tool with a small parameter set.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's potential for destructive actions (delete, reset) and lack of annotations or output schema, the description should provide more context such as safety warnings, when rules are required, or what status returns. The description is too minimal to fully prepare an agent for safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both 'action' and 'rule' with examples. The description adds no extra meaning about parameter semantics, and it omits the 'reset' action from its list, though the schema covers it. The baseline of 3 is appropriate because the schema does the heavy lifting, but the description fails to clarify the conditional requirement of 'rule' (not needed for status/reset).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a UFW firewall manager with a verb ('Manage') and specific resource ('UFW firewall rules'), and lists common actions in parentheses. It distinguishes itself from sibling tools like manage_service and docker. However, the verb 'manage' is somewhat generic and the action list omits 'reset', which appears in the schema, slightly reducing specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as run_command or manage_service. The description does not provide context for when this tool is appropriate, nor does it mention exclusions or prerequisites. The usage is only implicit from the sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_commandB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states what the tool does, but does not disclose potential side effects, privilege level, output format, or the risk of running arbitrary shell commands. This is a significant gap for a potentially destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no filler words. It is front-loaded with the core action and resource, and every word contributes to the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's broad and potentially dangerous nature (executing arbitrary shell commands), the description is incomplete. It lacks information about return values, exit codes, stderr handling, or whether the command runs with elevated privileges. The absence of an output schema makes the description even more essential, yet it provides almost no behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage with clear definitions for command, timeout, and working_directory. The description adds no extra meaning, but since the schema fully documents parameters, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Execute a shell command on the VPS via SSH' uses a specific verb (execute) and resource (shell command on VPS), clearly distinguishing it from sibling tools like read_file or list_services which are higher-level operations. The purpose is immediately obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus dedicated siblings such as docker, firewall, or manage_service. It does not mention that this is a general-purpose fallback or that specific tools should be preferred for their domains, leaving the agent without decision-making context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• system_statsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully bears the transparency burden, but it only states what the tool gets without disclosing side effects, required permissions, or output format. The verb 'Get' implies read-only behavior, but this is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that presents the primary purpose up front with no extraneous words. Every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, with one optional parameter, and the description covers its core function. However, it lacks any note about return format, platform constraints, or typical use cases, and the absence of an output schema makes the description the only source for such context. This is a minimally viable level of completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a complete description for the single 'include' parameter, including enum values and a default. The tool description's list of stats mirrors the enum options but adds no new semantic information beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function via a specific verb and resource ('Get system resource usage') and enumerates the categories (CPU, RAM, disk, network, uptime). This differentiates it from sibling tools focused on files, commands, services, and firewall.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or exclusions. The user must infer usage solely from the tool name and generic description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_serviceB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, but it does not mention side effects (e.g., start/stop alter service state), permission requirements, or output behavior. The listed actions imply mutation for some, but the description does not elaborate on what the agent should expect from status or logs compared to start/stop.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core purpose and lists actions parenthetically. It is concise with no wasted words, earning a perfect score for conciseness and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has multiple actions and no output schema, so the description should explain more about expected results and side effects, but it does not. It omits the reload and logs actions and fails to describe the behavior of each action or the response format, which is a significant gap for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all parameters with 100% coverage, so the baseline is 3. The description does not add additional meaning about parameter usage; it only lists a subset of the actions, omitting reload and logs, and does not explain the log_lines parameter's role.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: managing systemd services on a VPS, with an explicit list of common actions like start, stop, restart, status, enable, and disable. This distinguishes it from sibling tools such as run_command (generic command execution) and list_services (listing only). The verb 'manage' plus the resource and action examples provides a specific and unambiguous purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for systemd service operations on the VPS, but it does not explicitly state when to use this tool over alternatives or when not to use it. There are no exclusion criteria or references to sibling tools like docker or run_command, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_directoryA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It only states that it lists files and directories, but does not mention what information is returned (e.g., names only, permissions, sizes), whether hidden files are included by default, or any error conditions like nonexistent paths. The parameter descriptions in the schema cover these aspects, but the description itself adds no behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no unnecessary words. It front-loads the action and clearly states the resource and context, making it concise and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, but there is no output schema or mention of the return format. The description does not explain what the output looks like (e.g., a list of names, full details based on long_format), which is important for a tool without an output schema. It is adequate but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all parameters with descriptions (100% coverage), including path, long_format, and show_hidden. The tool description adds no parameter-specific meaning beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('List') and its resource ('files and directories'), with a specific path parameter. This distinguishes it from sibling tools like read_file (which reads file contents) and list_services (which lists services).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for exploring the filesystem, but it provides no explicit guidance on when to prefer this over alternatives like read_file or run_command. There is no mention of exclusions or alternative tools, leaving the context of use somewhat implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It states the tool reads file contents, but does not explicitly mention that it is read-only, has no side effects, or what happens in error cases. The straightforward nature of the operation gives it a baseline score, but more detail would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded and efficiently conveys the core purpose. Every word is necessary, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool of this simplicity, the description covers the essential purpose and the schema handles parameter details. It could mention that it is read-only or discuss error handling, but given the low complexity and no output schema, it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes both parameters (path required, max_lines optional), so schema coverage is 100%. The description adds no additional parameter context, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb and resource: 'Read the contents of a file on the VPS.' This distinguishes it from siblings like write_file and list_directory, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., write_file, list_directory). The description only states the purpose, not the context or conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing side effects and permissions. It fails to explicitly state that listing is a read-only operation, whether elevated privileges are required, or what output format to expect. The description adds no behavioral context beyond what the name implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately states the action and optional filter. No wasted words or redundant elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and high schema coverage, the description plus schema is nearly complete. It doesn't describe output format or error conditions, but for a straightforward read-only list, these are not critical gaps. A 4 reflects the slight lack of output/error documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: the filter parameter is fully described with its enum values and default. The description's 'filter by state' only restates the schema, adding no additional semantic meaning. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists systemd services with an optional state filter, which distinguishes it from siblings like manage_service (manages services) and list_directory (lists files). The verb 'list' and resource 'systemd services' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys the tool's primary use (listing services) and the available filter by state, giving clear context for when to use it. It doesn't explicitly exclude alternatives or mention when not to use it, but the sibling tools make the differentiation inferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly states 'creates or overwrites', which is the key behavioral trait, especially important given no annotations are provided. This discloses the destructive overwrite behavior, adding value beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero waste. It immediately communicates the action and the key nuance (creates or overwrites) without any redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write tool, the combination of description and fully-described schema is mostly complete. It covers the main purpose and behavior, though it could mention the default of create_dirs, but that is already in the schema. No output schema exists, so return values are not needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% description coverage for all three parameters, so the description does not need to add parameter meaning. The schema already explains path, content, and create_dirs adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Write content to a file on the VPS (creates or overwrites)' uses a clear verb (write) and resource (file on the VPS), explicitly distinguishing this tool from siblings like read_file and list_directory. It precisely states the action and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by its nature (write vs read), but provides no explicit guidance on when to use this tool over alternatives or any exclusion criteria. With siblings like read_file, the contrast is clear but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.