Server Quality Checklist

Tool Scores

• analyze_diff_impactB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full disclosure burden. It does not state whether the operation is read-only, what the return format is, whether it has side effects, or any permission requirements. It only describes the analysis action without behavioral detail, leaving the agent to infer safety and outcomes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-constructed sentence that front-loads the core action and purpose. Every word contributes to meaning, with no filler or redundancy. It is appropriately concise for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately conveys what the tool does and its input modes, but it lacks details on output format, expected results, limitations, or performance considerations. Given the tool's complexity (AST diff analysis), more context would help an agent know what to expect from the result and how to interpret it. The gap is not severe for a read-style tool, but it is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so all three parameters are already documented. The tool description adds minimal extra meaning—it references the diff scope (branches, commits, working tree) which aligns with the parameters but does not enrich the parameter understanding beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (analyze modified AST nodes) and outcome (determine affected downstream functions, classes, components), covering the core purpose. It is clear about the resource and scope, but it does not explicitly differentiate from the sibling tools get_repo_structure and inspect_blast_radius, which could potentially overlap in intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the input modes (between branches, commits, or uncommitted working tree), providing context for when it might apply, but it gives no explicit guidance on when to prefer this tool over its siblings or any conditions that would make it inappropriate. There is no mention of alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_blast_radiusB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states what the tool outputs (regression points, dependents, broken tests) but does not disclose behavioral traits such as whether it is read-only, whether it runs tests, requires special permissions, or has performance implications. It gives an outcome but no process or side-effect details, which is a moderate gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that lists the key outputs with zero filler. Every clause adds information about what the tool does, making it appropriately concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that analyzes blast radius, the description fails to mention the return format or structure (e.g., lists, counts, path groupings) despite having no output schema. It also omits any preconditions (e.g., repository must be initialized) or costs. Given the tool's complexity and lack of output schema, this is a notable completeness gap, though the core purpose is covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are already documented in the schema. The description adds no extra meaning beyond referencing the 'targeted file change', which aligns with targetFile. It neither enriches nor contradicts the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'identifies' and specifies the resource: a targeted file change. It enumerates concrete outcomes (regression points, direct/transitive dependents, broken test suites), which is specific. However, it does not explicitly differentiate this from the sibling analyze_diff_impact, so it's clear but not fully distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the siblings analyze_diff_impact or get_repo_structure. The description implies a use case (inspecting impact of a file change) but offers no exclusions, prerequisites, or alternative conditions. The agent is left to infer when this is the right tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repo_structureB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a meaningful behavioral trait—stripping implementation bodies to save tokens—but does not explicitly state that the operation is read-only, mention any side effects, or describe limitations. Some transparency is present, but key aspects remain uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose and then adds the pruning detail. It is concise, with no wasted words, and effectively communicates the tool's value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the output concept and the schema covers parameters, but there is no output schema and the description doesn't mention return format or usage context. Given the tool's simplicity, it is mostly complete, though it could benefit from a note on when to use it or what the skeleton structure looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all five parameters are fully documented in the schema. The description itself adds no parameter-level details beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a hierarchical, AST-pruned structural map with token-efficient code skeletons, which is specific to the resource and operation. It does not explicitly differentiate from sibling tools like analyze_diff_impact or inspect_blast_radius, but the purpose is unambiguous and distinct by nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools. It only describes what it does, without any conditions, exclusions, or references to alternatives, leaving usage context entirely to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.