Server Quality Checklist

Tool Scores

• execC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool executes commands but doesn't mention critical behaviors: whether this is a read/write operation, what permissions are needed, if commands run as root by default, what happens on timeout, or how output/errors are returned. For a command execution tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's appropriately sized for a tool with comprehensive schema documentation and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a command execution tool with 7 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns (stdout/stderr/exit code), error conditions, security implications, or interaction patterns. The combination of mutation capability and lack of behavioral context makes this inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema. Baseline 3 is appropriate when the schema does the heavy lifting for parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute') and target resource ('a command in a Docker container'), providing a specific verb+resource combination. However, it doesn't explicitly distinguish this from sibling tools like get_container_info or list_containers, which are read-only operations versus this execution tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., container must be running), when not to use it (e.g., for simple container inspection), or how it differs from sibling tools like get_container_info for container metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_container_infoA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it indicates this is a read operation ('Get detailed information'), it doesn't specify whether it requires specific permissions, how it handles errors (e.g., invalid container IDs), latency, or output format. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose and scope. It front-loads the core action ('Get detailed information') and lists specific details without unnecessary elaboration, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage (100%), the description adequately covers the basic purpose. However, with no annotations and no output schema, it lacks details on behavioral aspects like permissions, error handling, and return values. For a read-only tool, this is a moderate gap, making it minimally viable but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'container_id' documented as 'Container ID or name' in the schema. The description doesn't add any further meaning beyond this, such as examples or constraints (e.g., case sensitivity, partial matches). Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description provides no additional parameter insights.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get detailed information') and resource ('about a specific Docker container'), with explicit enumeration of the information types included (state, network, mounts, environment, resource configuration). It distinguishes from sibling tools like 'list_containers' (which likely lists containers rather than providing detailed info) and 'exec' (which executes commands).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'about a specific Docker container' and listing the detailed information returned, which suggests it's for inspecting a known container. However, it doesn't explicitly state when to use this tool versus alternatives like 'list_containers' (e.g., for overview vs. details) or 'exec' (e.g., for inspection vs. interaction), nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_containersA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the default behavior (shows only running containers) and how to change it (use all=true), which adds useful context. However, it lacks details on output format, pagination, or error handling, leaving some behavioral aspects unclear for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose ('List Docker containers') and efficiently follows with usage details in a single, clear sentence. Every part of the description earns its place by providing essential information without any waste or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is reasonably complete for basic usage. It covers the purpose and key parameter behavior. However, without annotations or an output schema, it lacks details on return values or potential errors, which could be important for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'all' parameter fully documented in the schema. The description adds value by explaining the default behavior and the effect of setting 'all=true', but this is largely redundant with the schema's description. Since schema coverage is high, the baseline score of 3 is appropriate, as the description provides some reinforcement but no significant additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('Docker containers'), making the purpose specific and unambiguous. It distinguishes this tool from sibling tools like 'exec' and 'get_container_info' by focusing on listing rather than executing commands or getting detailed information about individual containers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'By default shows only running containers, use all=true to show all containers including stopped ones.' This clearly defines the default behavior and how to modify it, offering practical usage instructions without needing to reference alternatives explicitly, as the context is straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.