Server Quality Checklist

Tool Scores

• tail_container_logsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains concurrency and the JSON map output for 'hosts' mode, but does not describe error handling, behavior for invalid hosts, or the exact output format for single-host mode.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary action, and contains no filler or repetition. Every sentence adds useful information about the tool's capabilities and preferred usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a moderate number of parameters and no output schema, but the description explains the return format for multi-host mode and clearly indicates the tool's purpose. Single-host output is implied to be raw log lines, and no annotations are present, but the tool is a non-destructive read operation, so the description is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage of parameter descriptions, so the baseline is 3. The description adds meaningful context by clarifying the relationship between 'host' and 'hosts' as alternative modes and explaining the multi-host output behavior, which goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the last N lines of logs from a remote Docker container, with a specific verb and resource. However, it does not explicitly distinguish itself from the sibling tool 'tail_log', so it lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need container logs) and gives guidance on using 'hosts' vs 'host', but it does not explicitly state when to use this tool versus alternatives like 'tail_log' or other log-related tools. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_processesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It explains the output format (JSON map for multiple hosts) and the effect of 'full_info', but does not mention safety (e.g., read-only nature), error handling, or permissions. The description is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose, and contains no redundant information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 100% parameter schema coverage, the description sufficiently covers the tool's functionality, including the two modes, regex matching, and optional stats. It lacks details on single-host output format and error cases, but overall is complete enough for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minor value beyond the schema: it explains that 'hosts' allows concurrent queries and returns a map, and implies the default output without full_info. However, much of the param info is already in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches running processes on one or multiple hosts using a regex pattern, and differentiates between single and multiple host modes. This specific verb+resource description distinguishes it from siblings like list_hosts or run_command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to prefer 'hosts' for finding services across multiple nodes, providing clear usage context. However, it does not explicitly mention when not to use this tool or alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_portsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description must provide all behavioral context. It covers basic behavior and output format for single/multiple hosts but omits error handling, permissions, or system impact details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently convey the main purpose and additional usage details, front-loading essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and three parameters, the description explains return types for both modes and optional filtering. It lacks details on default behavior (if no parameters) and error scenarios, but is fairly complete for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters 100%, so baseline is 3. The description adds value by clarifying the semantic difference between 'host' and 'hosts' and the output format, thus scoring above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists active listening TCP/UDP ports, processes, and PIDs. It distinguishes itself from siblings like list_hosts and search_processes by its specific function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains when to use 'host' vs 'hosts' and recommends 'hosts' for scanning multiple machines. It mentions optional port filtering but lacks explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tail_logA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that using 'hosts' returns a JSON map of hostnames to log output, but does not mention error handling, authentication needs, or behavior for missing files. Adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy. The first sentence states the core purpose, the second adds a usage preference. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters with 100% schema coverage and no output schema, the description covers the key behavioral differences and output format. It implicitly covers the 'lines' default (100 is in schema). Missing details on return format for single host, but schema suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are already documented. The description adds value by explaining the functional difference between 'host' and 'hosts' and the output format for multiple hosts, which is beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the last N lines of a remote log file, with options for single ('host') or multiple ('hosts') hosts. It distinguishes from siblings like 'tail_container_logs' (container-specific) and 'run_command' (arbitrary commands).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to prefer 'hosts' for querying logs across multiple machines, providing clear guidance on when to use each parameter. It does not explicitly state when not to use the tool, but the purpose is sufficiently scoped.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_hostsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It correctly indicates a read-only operation without side effects, but does not disclose authentication or rate limits. For a simple list tool, this is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no waste. The definition is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema tool with no annotations, the description fully covers purpose and usage. No further context is required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema coverage is 100% (none). Baseline for 0 params is 4; no additional parameter information is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a list of configured remote SSH hosts, with a distinct resource (remote hosts) that differentiates it from siblings like list_groups or list_ports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'useful to see what remote machines are available to target' implies the usage context, but it does not explicitly exclude cases or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_log_patternA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the transparency burden. It discloses blocking, streaming, and the JSON map return for multi-host usage, including success/error/timeout statuses. However, it does not detail the single-host return format or what happens on timeout for a single host, leaving a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action and no filler. The second sentence provides valuable use-case context, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core blocking behavior and multi-host return format, but it falls short on single-host return details and timeout semantics for a single host. Since there is no output schema, these missing details affect completeness for a wait tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaning by explaining the host vs hosts distinction, emphasizing concurrent execution, and describing the return map for hosts, which goes beyond the schema field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Blocks and streams a remote log file or Docker container logs on a single host or multiple hosts concurrently until a regex pattern is matched or a timeout is reached,' clearly stating the specific verb, resource, and conditions. It distinguishes itself from sibling tools like tail_log and tail_container_logs by emphasizing the blocking/waiting behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says it is 'Extremely useful for verifying startup or events across cluster nodes without polling,' giving clear usage context. It also explains the difference between single-host and multi-host use, but it does not explicitly mention alternatives or when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_commandA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It does well by explaining the return format for hosts (JSON map with stdout, stderr, exit codes), background execution (returns log path immediately), quiet mode, and output abbreviation. It does not cover potential side effects or permissions, but for a command runner, this is reasonably transparent and much more than a minimal description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first defines the core action, the second details the return format for multi-host, and the third summarizes optional features and a usage preference. No redundant or filler content; it is front-loaded with the most critical information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex (8 parameters, no output schema). The description covers the main behaviors: host vs. hosts, return format for hosts, background, quiet, and abbreviation. It does not explicitly state the return format for a single host, but that is inferable from the context. Given the schema's thorough field descriptions, the overall description is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the relationship between 'host' and 'hosts' (single vs. concurrent), and the return format when 'hosts' is used—information not in the schema. It also clarifies the purpose of background mode and output abbreviation, complementing but not repeating schema fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes a shell command on a host or hosts, using specific verbs and resources. It distinguishes itself from siblings like list_hosts and tail_log, which are for querying or following logs, not running commands. The dual mode (single vs. multiple) is explicit and immediately understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: to run shell commands on hosts. It also gives an explicit preference: 'Prefer 'hosts' to execute commands across cluster nodes simultaneously.' This guides selection between host and hosts modes. However, it doesn't explicitly mention when NOT to use this tool or name alternative tools for similar tasks, leaving a small gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_system_statsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It discloses concurrent queries via 'hosts' and the return format (JSON map), but omits details like rate limits or auth requirements. Still, it's fairly transparent for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first summarizes purpose and parameters; the second adds guidance. No fluff, front-loaded with key info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description covers return behavior for 'hosts' and implies metrics for 'host'. It addresses all parameters and provides usage context, making it complete for a tool with two optional params.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds value by explaining the return format difference between 'host' and 'hosts' and the concurrency behavior, surpassing what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches CPU, RAM, and disk utilization metrics, distinguishing between single host ('host') and multiple hosts ('hosts'). This differentiates it from siblings like list_hosts or search_processes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends preferring 'hosts' for parallel cluster queries, guiding when to use each parameter. It lacks explicit when-not-to-use or alternatives beyond the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_groupsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It clearly explains the return format (map of group names to host lists), which is important behavioral context. It doesn't mention side effects, but for a read-only listing tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and every sentence adds value. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a zero-parameter, no-output-schema tool with low complexity. The description fully explains what it returns and why it's useful, making it complete for an agent to select and invoke.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the baseline for 0 params is 4. The description doesn't need to explain parameter semantics since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a map of configured remote SSH host groups, with specific details about keys and values. It differentiates from siblings like list_hosts by focusing on multi-host groups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says it's useful to see what multi-host SSH groups are available to query/run/watch, implying when to use it. However, it does not explicitly mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.