Server Quality Checklist

Tool Scores

• improve_codeC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided, so the description must carry the full burden of behavioral disclosure. It fails to mention that this tool modifies code files, may commit changes (auto_commit param), or has any side effects, leaving the agent without critical safety information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words, front-loading the core purpose. It is concise and well-structured, though this brevity contributes to gaps in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the rich schema and output schema, the description omits essential context for a code-modifying tool. There is no mention of side effects, commit behavior, or filesystem interaction, and with no annotations, the description is insufficient for safe use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all five parameters (files, model, auto_commit, instruction, project_path). The description adds no parameter-specific meaning beyond the general purpose, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('improve code quality, fix bugs, or implement features') using Aider, which gives a specific verb and resource. However, it overlaps with sibling tools like fix_bug, add_feature, and refactor without distinguishing itself, so it doesn't fully differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description even includes 'fix bugs' and 'implement features', which are covered by sibling tools, but it does not explain when to choose this tool or when to choose a sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fix_bugC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It only mentions 'using Aider,' which implies an external dependency but doesn't state that the tool modifies code, may require git access, or could have side effects on the file system. There is no mention of permissions, reversibility, or failure modes. This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at six words with no wasted text. However, it is so brief that it lacks substance, which hurts other dimensions. For conciseness alone, it earns credit for being efficient, but it is not structured or informative enough to be a model of good tool descriptions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, mutation behavior) and lack of annotations, the description is far from complete. It does not explain what Aider is, how the bug will be fixed, which files are affected, or what the output will be. The presence of an output schema reduces the need to describe return values, but the description still leaves major gaps about usage and effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters have descriptions. The tool description itself adds no additional parameter semantics beyond what the schema already provides. Baseline of 3 is appropriate since the schema does the heavy lifting and no extra context is given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The verb 'fix' combined with resource 'bug' clearly indicates the tool's purpose. It distinguishes from siblings like add_feature, refactor, and improve_code by targeting bugs specifically. However, 'using Aider' may be unclear if the agent doesn't know Aider, and it lacks additional context about the scope or manner of fixing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It merely states what it does, without specifying conditions (e.g., 'use when there is a bug') or exclusions (e.g., 'for new functionality use add_feature'). No naming of sibling tools or decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refactorC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden of behavioral disclosure. It only mentions 'using Aider', hinting at an external tool, but does not explain side effects (e.g., modifying files), required permissions, or how the refactoring is executed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. However, it is under-specified for a tool of this complexity, so while concise, it is not fully appropriate in length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and sufficient parameter documentation, the description lacks essential context: when to use it, what behavior to expect, and how it differs from sibling tools. The agent is left without enough information to select or invoke this tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameters are already well documented in the input schema. The description adds no additional semantic meaning beyond what the schema provides, which meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('refactor') and names the resource ('code'), making the core action clear. However, it does not differentiate from sibling tools like improve_code, and 'using Aider' adds an implementation detail rather than clarifying scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives such as improve_code, add_feature, or fix_bug. There is no mention of prerequisites, exclusions, or context in which refactoring is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. The word 'Get' implies a read-only operation and no side effects, but this is not explicitly stated, and the tool's behavior (e.g., whether it checks for changes, current branch, or commits) is undisclosed. The output schema exists but the description does not reference it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that captures the tool's essence without wordiness. Every word contributes to the meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status-checking tool with one parameter and an output schema, the description is minimal but arguably sufficient for a basic understanding. However, it lacks context about what 'Aider and git status' specifically reports, and the absence of usage guidance makes it less complete for an agent deciding when to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter project_path is fully documented in the schema with a clear description and default value. The description does not add extra detail, but with 100% schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly uses the verb 'Get' with the resource 'Aider and git status', making the action unambiguous. It is distinct from sibling tools which are code modification actions (improve_code, add_feature, etc.), though it does not explicitly describe what 'Aider and git status' entails beyond the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The sibling tools are all action-oriented, so one might infer this is for pre-action status checks, but the description does not state this or offer any alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_featureB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral disclosure burden. It only says 'Add a new feature using Aider,' which implies mutation but gives no detail on side effects, permissions, or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no wasted words. It is efficiently concise, even if it sacrifices depth for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite full schema coverage and an output schema, the description omits crucial context: when to use add_feature versus improve_code or fix_bug, and what using Aider entails behaviorally. The tool is complex enough that this minimal description leaves major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with all four parameters having descriptions. The tool description adds no parameter semantics, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The verb 'Add' and resource 'feature' clearly state the action. Adding 'using Aider' provides method context, distinguishing it from siblings like improve_code, fix_bug, and refactor.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool over its siblings. There is no mention of prerequisites, alternatives, or contexts where other tools would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.