Server Quality Checklist

Tool Scores

• inspect_memory_dumpA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explains the non-destructive nature, the use of docker stats, and the optional inclusion of process list. It does not disclose permissions or rate limits, but for a read-only tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a short summary, then usage notes, then a structured Args section, and finally Returns. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only 2 parameters, no output schema, and no annotations, the description is quite complete. It covers inputs, outputs, and behavioral details. Could mention error handling for non-running containers, but that's minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It clearly describes 'container_id' as the container ID or name, and 'include_processes' with its effect (runs ps aux). This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns live memory, CPU stats, and top processes for a container. The verb 'inspect' combined with the resource 'memory_dump' is apt, and the description distinguishes this tool from siblings like 'execute_isolated_script' and 'stream_container_logs'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes the container must be running and that it uses a point-in-time snapshot. It also clarifies that it's not a real heap dump, which prevents misuse. However, it does not explicitly state when to use this tool vs the siblings, though the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_isolated_scriptA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behaviors: validates container_id, blocks dangerous patterns, enforces a hard timeout via asyncio, and returns a dict with stdout, stderr, exit_code, timed_out. This informs the agent of safety and timeout guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, starting with a clear purpose sentence followed by a bulleted Args and Returns list. Every sentence is informative and no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity (3 parameters, no output schema, no annotations), the description is thorough: covers purpose, safety, timeout behavior, and return format. The agent has enough information to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides clear explanations for all three parameters: container_id is ID or name, command is executed via sh -c, timeout_seconds is max seconds before killing. This adds critical meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run a shell command string') and the resource ('inside a container via docker exec'). It is distinct from sibling tools (inspect_memory_dump and stream_container_logs) which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for running shell commands but does not explicitly specify when to use this tool versus alternatives, nor does it mention when not to use it. No comparison to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stream_container_logsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly explains the parameters and the return value, implying a read-only operation ('fetch'). It could explicitly state that the tool does not modify state, but the information is sufficient for safe usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: a single purpose sentence, followed by well-structured Args and Returns sections. No wasted words, and the most important information (purpose) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, the description provides a complete return structure (container_id, log_lines, line_count, truncated). All parameters are explained. The tool is simple and the description covers all necessary information for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the schema alone provides no parameter descriptions. The description fully compensates by explaining each parameter's purpose (container_id as ID or name, tail as number of lines, since as ISO timestamp, filter_keyword as substring). This adds essential meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch stdout/stderr logs from a Docker container by ID or name.' This clearly identifies the verb (fetch), the resource (logs), and the scope (by container ID/name). It is distinct from sibling tools which deal with scripts and memory dumps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool (to fetch logs) and explains each parameter's role. However, it does not explicitly state when not to use it or provide alternatives, though siblings are sufficiently different that exclusion is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.