Server Quality Checklist

Tool Scores

• extract_functionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'extract' implies a mutation operation, the description doesn't specify whether this modifies the original file, creates a new file, requires specific permissions, or has any side effects. No information about error conditions, rate limits, or response format is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for the tool's complexity and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 7 parameters and no annotations or output schema, the description is inadequate. It doesn't explain what happens after extraction, whether the original code is replaced with a function call, what the return value contains, or any error conditions. The description fails to compensate for the lack of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain relationships between parameters (like how start/end positions work together) or provide usage examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('extract') and target ('selected code into a new function'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'move_function' or 'rename_symbol', which might involve similar code manipulation operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'move_function' or 'rename_symbol'. There's no mention of prerequisites, appropriate contexts, or exclusions for when this tool should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_referencesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions using LSP, which hints at language-aware analysis, but doesn't describe key traits like whether it's read-only (implied by 'find'), performance considerations (e.g., speed, network usage), error handling, or output format. For a tool with 4 parameters and no annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Find all references to a symbol') and adds technical context ('using LSP'). There is no wasted verbiage or redundancy, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, no output schema), the description is incomplete. It lacks details on behavioral aspects (e.g., read-only nature, potential side effects), output expectations (what 'references' look like), and usage context relative to siblings. Without annotations or output schema, the description should compensate more to provide a complete understanding, which it does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (file, line, character, language) with descriptions and defaults. The description adds no additional meaning beyond what the schema provides, such as clarifying how parameters interact (e.g., file-path format, line/character indexing) or usage examples. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Find all references') and the resource ('to a symbol'), specifying it uses LSP (Language Server Protocol). It distinguishes from siblings like 'rename_symbol' or 'move_function' by focusing on reference discovery rather than modification. However, it doesn't explicitly contrast with sibling tools beyond the general purpose difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies usage for locating symbol references via LSP, but it doesn't specify scenarios (e.g., code navigation vs. refactoring), prerequisites, or when not to use it (e.g., for non-symbol elements). Sibling tools like 'extract_function' or 'rename_symbol' suggest related contexts, but no comparative advice is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rename_fileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'LSP rename capabilities' which hints at Language Server Protocol integration, but doesn't disclose critical behavioral traits: whether this requires specific permissions, if it's destructive (renaming typically is), what happens on failure, or any rate limits. For a mutation tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, stating the core purpose immediately. Every word earns its place without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns, error conditions, or behavioral implications. The mention of 'LSP rename capabilities' adds some context but doesn't compensate for the missing safety and response information needed for a destructive operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema. It doesn't explain parameter relationships, constraints, or usage patterns. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('rename') and resource ('file or folder'), and specifies the mechanism ('using LSP rename capabilities'). However, it doesn't explicitly differentiate from sibling tools like 'rename_symbol' or 'move_function', which appear related. The purpose is specific but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'rename_symbol', 'move_function', and 'find_references', there's no indication of when this tool is appropriate versus those other options. No prerequisites, exclusions, or contextual usage information is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• move_functionB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'move' implies mutation and 'LSP refactoring' suggests editor integration, it doesn't address critical behaviors: whether this operation is reversible, what happens to imports/references, if it requires specific permissions, or what the output looks like. For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and includes the implementation mechanism. Every element earns its place without redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 6 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what happens after the move, how errors are handled, or what the agent should expect as a result. The combination of mutation complexity and lack of structured metadata requires more descriptive context than provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter semantics beyond what the schema provides (like explaining why both line and character are needed for precise location). The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Move a function') and mechanism ('using LSP refactoring'), distinguishing it from siblings like extract_function (which creates new functions) and rename_symbol (which renames within the same file). It precisely identifies both the resource (function) and destination (different file).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like extract_function or rename_symbol. It doesn't mention prerequisites, constraints, or typical scenarios for moving functions versus other refactoring operations, leaving the agent to infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rename_symbolB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool renames symbols 'across all files', which suggests a potentially destructive, wide-scope operation, but doesn't disclose behavioral details like whether changes are reversible, what permissions are needed, how conflicts are handled, or what the response looks like. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('rename a symbol') and adds clarifying scope ('across all files'). There is zero wasted text, and it's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks critical behavioral context (e.g., safety, permissions, response format) and doesn't compensate for the absence of structured data. The 100% schema coverage helps with parameters, but overall completeness is poor for a tool that modifies code across files.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters (file, line, character, newName, language) with descriptions. The description adds no additional parameter semantics beyond what the schema provides, such as explaining how 'character' is used or what 'language' affects. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'rename' and the resource 'symbol', specifying it applies to variables, functions, classes, etc. across all files. It distinguishes from siblings like 'rename_file' (which renames files, not symbols) and 'move_function' (which moves rather than renames).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for renaming symbols in codebases, but doesn't explicitly state when to use this vs. alternatives like 'find_references' (for locating symbols) or 'extract_function' (for refactoring). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.