Server Quality Checklist

Tool Scores

• execute_bashA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It does add value by noting the execution directory and the return format (stdout, stderr), but it omits critical details such as side effects, environment dependencies, error handling, or potential timeouts. For a command-execution tool, more transparency is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, using a standard docstring format with Args and Returns sections. Every sentence serves a purpose, and there is no fluff. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basics: what it runs, where it runs, and the return type. However, given the tool executes arbitrary shell commands in a shared working directory, more context is needed about the relationship with 'set_cwd', potential side effects, and how command failures manifest. The lack of annotations and output schema increases the burden, but the description provides just enough for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for 'cmd', so the description's 'The shell command to execute' provides some semantic clarification. However, this is minimal and largely restates the parameter name, adding little beyond what one might infer from 'cmd'. With 0% schema coverage, the description only partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a bash command and specifies the scope ('global working directory'), which gives a specific verb and resource. However, it does not explicitly differentiate from the sibling tool 'set_cwd', though the focus on execution vs. setting the directory is implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (run bash commands in the global working directory) but provides no explicit guidance on when to use this tool versus alternatives like 'set_cwd'. There is no mention of prerequisites, exclusions, or when not to use it, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_cwdA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states that it sets the global working directory and returns a confirmation message, but it does not disclose potential errors, validation behavior, or whether the change is session-wide. This is minimal but adequate for such a simple state-changing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a one-sentence purpose, a clear Args section, and a Returns section. Every line serves a purpose with no unnecessary fluff. The structure is front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers the essential aspects: what it does, the parameter requirement (absolute path), and the return type. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only the parameter name and type (string), with 0% coverage. The description compensates by explicitly stating 'path: The absolute path to use as the new working directory', adding the important constraint that the path must be absolute. This adds meaningful semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Set the global working directory for bash commands', which clearly identifies the verb (set), resource (global working directory), and scope (for bash commands). This effectively distinguishes it from the sibling tool execute_bash, which runs commands rather than changing the working directory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool should be used to change the working directory for subsequent bash commands, but it does not explicitly state when to use it versus execute_bash or any exclusions. The usage context is clear but not formally articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.