Server Quality Checklist

Tool Scores

• list_dirB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It indicates a read-only listing operation, which is inherently non-destructive. However, it does not mention permissions, error handling, or any side effects, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It is front-loaded and efficiently conveys the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and the presence of an output schema (for return format), the description is minimally complete. However, it lacks details about hidden files, sorting, error conditions, or path handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the parameters (host, path) beyond their names. While the names are somewhat self-explanatory, the description adds no additional semantic context about expected values or formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists directory entries over SFTP, with a specific verb and resource. It implicitly differentiates from sibling tools like read_file (file content) and run_command (command execution), though it doesn't explicitly compare.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives, nor are there prerequisites or exclusions provided. The description lacks any when-to-use or when-not-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_commandA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool does not execute the command (non-destructive), returns a verdict, reason, and matched rules. However, it does not mention any side effects or whether it requires specific permissions, but for a dry-run tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the key action and outcome. Every sentence adds value with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (implied from context signals), the description provides adequate high-level information about return values. It could mention pagination or error cases, but for a simple check tool this is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the parameters 'host' and 'command'. While the names are somewhat self-explanatory, the description should add context like format or examples to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: dry-run safety policy without executing. It specifies the verb 'check' and resource 'command safety policy', and distinguishes from sibling run_command by contrasting execution vs dry-run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to understand why a command would be refused before calling run_command.' This provides clear guidance on when to use this tool and implies not to use it when you intend to execute the command.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses key behaviors: reading over SFTP, size cap via max_bytes, and default policy. However, it omits error handling, authentication needs, and handling of missing files or binary content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two paragraphs: a one-sentence summary followed by a concise argument list. Every sentence adds value, no fluff, and the purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (which explains return values), the description covers purpose, parameters, and behavior adequately. It lacks guidance on error conditions or security, but for a simple read tool it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema property descriptions are missing (0% coverage), the description adds meaning by explaining each parameter: host as alias, path as absolute/relative, max_bytes with default behavior. This compensates well, though some details (e.g., host alias format) are absent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (Read), resource (remote file), method (over SFTP), and constraint (size-capped). It distinguishes this tool from siblings like list_dir and run_command by specifying its exact function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for reading files, but does not explicitly state when to use alternatives (e.g., list_dir for directory listings) or when not to use it. No prerequisites mentioned, but the purpose is clear enough for basic selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_commandA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It details the behavior: subject to safety policy, possible statuses (ok, needs_confirmation, blocked, error), and how to handle destructive commands. It could mention more about the safety policy, but it's adequately transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a main sentence, a bullet list for result status, and a numbered list for parameters. It is concise yet comprehensive, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, output schema exists), the description covers all essential aspects: host source, command execution, status handling, and parameter details. It explains the output structure indirectly by mentioning exit_status, stdout, stderr, which is sufficient for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides rich semantics for each parameter beyond the schema: host must be an alias (not raw), command is a single shell command, confirm authorizes previously flagged commands, and timeout has a default and per-command context. With 0% schema coverage, this fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Run one shell command on an SSH host', which is a specific verb-resource pair. It distinguishes from sibling tools like check_command and list_hosts by specifying that the host parameter must be an alias from list_hosts, and outlines the output structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool, including that host must be an alias from list_hosts, and explains the result statuses and how to handle the 'needs_confirmation' status with a confirmation flow. It lacks explicit exclusion of other cases but is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_hostsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description explains the output usage (aliases for other tools) and implies a read-only operation. It does not detail side effects or idempotency, but the context is sufficient for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose and immediate usage guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, output schema exists, and sibling tools are provided, the description fully covers the tool's role and integration with other tools. It tells the agent exactly when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. The description adds value by explaining the purpose of the output, which goes beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists SSH hosts and active safety policy, with a specific verb and resource. Differentiates from siblings by indicating it's the first call to provide aliases for other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this first' and instructs to use returned alias values as host argument for other tools. Also warns against guessing hostnames, providing clear when-to-use and what not to do.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.