Server Quality Checklist

Tool Scores

• shell_configA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description serves as the only behavioral disclosure. It implies a read-only operation ('Return') but does not explicitly state that it has no side effects or that it is safe. It does clarify that secrets are excluded, which is a useful constraint. However, it does not mention any authentication or permission requirements, which could be relevant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action ('Return') and specifies the resource. There is no wasted verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there are no parameters, the description sufficiently conveys the tool's function. The existence of an output schema covers return details, so the description need not elaborate. It does not mention usage context, but that is already addressed in usage_guidelines; for a simple config retrieval, this is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description is not required to explain parameter syntax. The phrase 'non-secret' adds context about the output scope but does not relate to parameters. For a parameterless tool, this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and a clear resource ('effective non-secret shell policy configuration'). It distinguishes itself from the sibling tool by nature (configuration retrieval vs. execution), though it does not explicitly name the alternative. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to use this tool versus the sibling shell_execute. An agent must infer that this is for reading configuration, not executing commands. The description does not mention any prerequisites, alternatives, or context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shell_executeA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It mentions that shell syntax is not interpreted, that the timeout is capped at 600 seconds, and that directory paths must resolve inside WORK_DIR if configured. However, it does not warn about the potentially destructive nature of executing arbitrary commands or the side effects of the executed process, which is a notable omission for a command execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a clear one-line summary, a detailed 'Args' section, and a 'Returns' section. It is appropriately detailed for a tool with multiple parameters but remains readable. The example for the command parameter adds clarity without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential invocation details: command format, directory constraints, timeout cap, and the return fields (stdout, stderr, exit status, etc.). It does not explain prerequisites such as the meaning of WORK_DIR, but this is likely a global configuration rather than tool-specific. Given the presence of an output schema (though not shown), the return-field listing is sufficient. Overall, it provides enough for an agent to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides no descriptions for any of the four parameters (schema coverage 0%), so the description must fully compensate. It does so thoroughly by explaining each parameter: the command array with examples, optional stdin, directory restrictions relative to WORK_DIR, and timeout cap. This adds substantial meaning beyond the schema's type definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Execute one command as an argv array.' It clearly distinguishes from the sibling tool 'shell_config' by implying execution vs. configuration, and provides a concrete example to clarify the argv format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear guidance on how to invoke the tool, including the requirement to pass an argv array rather than shell syntax, and clarifies timeout and directory constraints. However, it does not explicitly state when to prefer this tool over shell_config, though the purpose difference is evident from the name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.