Server Quality Checklist

Tool Scores

• rename_symbolA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It states that it renames the symbol 'and all its references', which is transparent about scope. However, it does not disclose potential side effects like handling name collisions, updating imports, or requiring confirmation. This is adequate but not fully comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with the main action in the first sentence. The example is helpful and well-structured. No superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is an output schema (though not shown), so the description need not explain return values. The description covers the core behavior well. However, given no annotations, it could add more context about prerequisites or limitations, but overall it is fairly complete for a rename tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The example explains the use of filePath, oldString, and newString, but the optional 'line' parameter is not described. The description adds meaning for three of four parameters via example, but missing explanation for 'line' leaves a gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'renames' and the resource 'symbol (function, variable, class, etc.) and all its references across the workspace'. It distinguishes from sibling tools which are about indentation, moving strings, and outlining, none of which are rename operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that it uses text-based matching rather than coordinate counting, which helps the agent understand when to use this tool (for text-based symbols) versus alternatives. It does not explicitly state when not to use it, but the example provides clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• outline_fileA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It discloses the output format (LINE_NUMBER:LINE_CONTENT, 1-indexed) and the specific selection criteria for lines. It does not mention destructive actions or auth needs, but as a read-only tool, that's acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. First sentence states purpose, then lists what is shown, then output format, then use case. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no annotations, the description covers the output comprehensively. It explains selection criteria and format. It does not mention limitations or error handling, but it's still fairly complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the 'filePath' parameter. While the parameter name is self-explanatory, the description adds no additional meaning or constraints (e.g., expected format, restrictions) beyond the schema's type declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a high-level outline of a file, similar to VS Code's folded view. It specifies what structural lines are included (class/def, separator comments). The purpose is distinct from sibling tools (indent_dedent, move_string_in_file, rename_symbol).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says it's 'useful for understanding the structure of a large file before reading it in full,' which implies when to use. It doesn't explicitly say when not to use, but the purpose is clear enough that an AI agent can infer appropriate usage contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• move_string_in_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that whitespace is ignored when matching, that indentation is not altered, and that the tool reports on blank lines after editing. It also describes the marker system for specifying cut/paste boundaries. However, it does not explicitly state that the operation modifies the file destructively (cutting lines), which could be implied but not clearly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is verbose and includes two long examples, which are helpful but detract from conciseness. The marker explanation is repeated twice. The front-loading is acceptable, but the description could be shorter while retaining clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, 0% schema coverage, no annotations, but has output schema), the description is fairly complete. It covers the marker system, paste positioning relative to newlines, and edge cases (single line vs. multi-line cut). It mentions the post-editing blank line report. However, details about the return value or error handling are absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning beyond the schema. It explains cutStartAt, cutEndAt, and pasteAt via examples, including the use of the ⬥ marker for multi-line contexts. It also mentions the optional pasteFilePath for moving between files. The description provides significant context that the schema alone does not convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool moves a continuous range of lines from one location to another within a file or between files. It distinguishes itself from sibling tools like indent_dedent (indentation) and rename_symbol (renaming) by specifying its unique operation—cutting and pasting lines based on substring matching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides two detailed examples illustrating typical use cases (moving a definition before another, moving a paragraph). It also advises to use indent_dedent for indentation adjustments after moving, giving clear guidance on when to use this tool versus alternatives. However, it does not explicitly state when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• indent_dedentA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behaviors: substring matching with ⬥ marker, ignoring trailing whitespace, leaving empty lines unchanged, and indenting/unindenting non-empty lines. While it does not cover error cases or persistence details, it provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with bullet points and an example, front-loading the purpose. It is appropriately sized for the complexity of the tool, though it could be slightly more concise by removing the common pattern commentary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, description does not need to cover return values. It explains the main parameters and behavior adequately. However, it lacks error handling or edge cases (e.g., substring not found), slightly reducing completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must add meaning. It explains the role of indentStartAt and indentEndAt as substring markers, and the indentDelta parameter via example (indentDelta=1). It does not fully explain indentSize or the sign of indentDelta, but it compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Indents or unindents a range of lines in a file', specifying the verb (indent/unindent) and resource (lines in a file). This clearly distinguishes it from sibling tools like move_string_in_file or outline_file, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to specify the range using substring matching with the ⬥ marker, and provides a common pattern example with a follow-up step. It gives clear context for usage, though it lacks explicit exclusions or when-not-to-use scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.