Server Quality Checklist

Tool Scores

• runC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only says 'return output' but does not mention how stderr/stdout are handled, exit codes, side effects, environment variables, or whether the command runs in a sandbox. This is a significant gap for a shell execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded, and contains no fluff. It is concise but arguably too terse to be fully helpful. Every word earns its place, though more content would be beneficial.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple but has nuanced behavior (output format, timeout implications, working directory) and an important sibling relationship with run_background. The description does not explain these contextual aspects, and there is no output schema to clarify what is returned. Leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameter descriptions already document command, cwd, and timeout. The description adds no additional semantic value beyond the schema. Baseline 3 is appropriate because the schema carries the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Identifies the action as executing a shell command and returning output. The verb 'execute' and resource 'shell command' are specific, and 'return output' implies a foreground/synchronous operation, distinguishing it from the background sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance about when to use this tool versus alternatives like run_background. Does not state that this runs in the foreground, blocks for completion, or is appropriate for commands that need immediate output. Lacks any context about selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_backgroundB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose side effects. It states the command is run in background but does not mention that it executes arbitrary system commands, whether it returns immediately, how output is handled, or what resources are consumed. This lack of transparency is risky for a command execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that directly states the tool's purpose with no redundant words. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations or output schema, the description is too thin. It does not explain the background execution model (immediate return?), how to monitor or retrieve output, or implications of running arbitrary commands. Simple but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage with descriptions for all three parameters (command, name, cwd), so the baseline is 3. The description adds no additional parameter meaning, but the schema already defines them well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Run') and resource ('command in background'), clearly distinguishing it from sibling tools (run, kill_background, list_background). It conveys exactly what action is performed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case (commands that should run in the background) but does not explicitly state when to use this tool over alternatives like 'run' or how it relates to kill/list_background. No exclusions or alternative names are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kill_backgroundA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states the action and target, but does not disclose side effects (e.g., irreversible process termination), behavior when the process is not found, or whether it uses a graceful shutdown. This is insufficient for a destructive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no unnecessary words. It is appropriately concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-parameter tool, the description covers the core action, but it lacks context about return values (no output schema) and edge-case behaviors. Given the destructive nature, additional guidance would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the 'name' parameter with a clear description, so the description's phrase 'by name' adds no new information. Baseline 3 applies due to 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Kill') with a clear resource ('background process') and a qualifier ('by name'). It clearly distinguishes from sibling tools like run and list_background, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit usage guidance is provided. The context implies it's for stopping background processes, but the description does not mention alternatives or exclusions, so usage must be inferred from the name and sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_backgroundA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly implies a read-only operation via 'list', but it does not disclose details such as the return format, whether it includes process IDs, or any side effects. For a simple list operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's function without any unnecessary words. It is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description is nearly complete. It clearly names the operation and scope, but it does not specify what the response contains (e.g., process names, IDs), which could be helpful. Given the tool's simplicity, this minor gap is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, and the schema description coverage is 100%, so the baseline is 4. The description adds no parameter details, but none are needed because there are no parameters to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states exactly what the tool does: list all running background processes. The verb 'list' is specific and clearly distinguishes it from sibling tools like run, run_background, and kill_background, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for when you want to see running background processes, but it does not explicitly contrast with sibling tools or state when not to use it. There is no mention of alternatives or exclusions, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.