Server Quality Checklist

Tool Scores

• get_code_actionsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral transparency. It discloses the file-open prerequisite but fails to state whether the operation is read-only or whether it applies changes, nor does it describe the return format. This leaves ambiguity about the tool's side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences and stays focused. The first sentence defines the purpose, the second adds illustrative examples, and the third states a key prerequisite. It is concise without being overly terse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description should explain what the tool returns. It mentions 'obtain available refactorings' but does not describe the response structure or possible error conditions. The prerequisite helps, but completeness is only moderate for a tool with six required parameters and no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all six parameters are individually documented in the schema. The description adds only the concept of a 'specific range', which aligns with the start/end line/column parameters, but provides no additional parameter semantics beyond what the schema already offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get code actions for a specific range in a file.' It provides a specific verb-resource pair and concrete examples (refactorings, quick fixes), distinguishing it from sibling tools like get_completions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this tool to obtain available refactorings, quick fixes, and other code modifications' and warns that the file must be opened first. It does not list exclusion criteria or alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_completionsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses an important behavioral constraint: the file must be opened first. It also clarifies that completions are context-based. However, it does not describe return format, error conditions, or whether an LSP server must be running, so it only partially covers transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is composed of five sentences but contains redundancy: sentences 1 and 2 both describe retrieving completions, and sentence 3 ('Helpful for...') adds little. The useful unique information (use case and prerequisite) could be expressed in fewer words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a completion tool, the description covers the core purpose, context, example use case, and a critical prerequisite (open file). It omits details about the return value or error handling, but given the simplicity and lack of output schema, it is reasonably complete for an agent to select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all four parameters with 100% coverage, so the baseline is 3. The description indirectly references location and context but does not add specific parameter details beyond what the schema provides. The prerequisite about opening the file adds a bit of context for file_path.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the main function ('Get completion suggestions at a specific location in a file') and elaborates on the types of completions (variable names, function calls, object properties). This distinguishes it from siblings like get_diagnostics or get_code_actions, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit use cases ('when determining which functions you have available in a given package, for example when changing libraries') and a prerequisite ('Requires the file to be opened first'). However, it does not explicitly name alternative tools or say when not to use it, so it is not a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_documentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the resource-freeing behavior, which is helpful, but doesn't state what happens if the file isn't open or whether unsaved changes are affected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core action, and every sentence adds value—purpose, usage context, and best practice. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description adequately covers purpose and usage. It omits return value/error scenarios, but these are not critical for a straightforward close operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage with a single parameter (file_path) and its description. The tool description adds no further semantic detail beyond the schema, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Close a file in the LSP server' with a specific verb and resource, distinguishing it from sibling tools like open_document and save_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit context for when to use the tool ('when you're done with a file to free up resources') and specific scenarios (long-running sessions, large codebases). It doesn't mention alternatives but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_info_on_locationA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the prerequisite 'Requires the file to be opened first' and implies a read-only operation via 'get information' and 'LSP hover.' It does not explicitly state there are no side effects or describe error behavior, but the key requirement is covered, making it reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat verbose and redundant. Sentences 2 and 3 both mention retrieving type information and documentation, and sentence 4 repeats 'use this' guidance. It could be condensed to two or three clear sentences without losing meaning, so it does not earn full points for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only hover tool with four parameters, the description covers purpose, usage scenarios, and a key prerequisite. There is no output schema, so the description gives a general idea of the return content ('detailed type information, documentation...'). While more detail on return format or error handling would improve completeness, it is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage for all four parameters, so the baseline is 3. The description does not add any parameter-specific details beyond the schema. It refers to 'a specific location' (line/column) and 'a file,' but these are already implied by the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Get information on a specific location in a file via LSP hover,' which is a specific verb+resource statement that clearly conveys the tool's function. It also distinguishes this from siblings like get_completions and get_code_actions by focusing on retrieving hover/type information rather than generating completions or actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context, stating it is 'particularly useful for understanding variable types, function signatures, and module documentation.' It also says 'Use this whenever you need to get a better idea on what a particular function is doing in that context.' However, it does not explicitly mention when not to use it or name alternative tools, so it misses the top score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restart_lsp_serverA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It mentions optional reinitialization with a new root directory and clarifies that the server won't auto-initialize if root_dir is omitted. However, it does not disclose side effects like loss of unsaved state, impact on open documents, or required permissions, which are important for a restart operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, with the core action in the first sentence. It is succinct, front-loaded, and contains no filler or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool without an output schema, the description covers the main purpose, common use cases, and parameter behavior. Minor gaps exist (e.g., side effects), but overall it provides enough context for an AI agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (root_dir is fully described in the schema). The description adds minor context by explaining that root_dir is optional and controls reinitialization, but it doesn't significantly go beyond the schema's own explanation. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Restart the LSP server process,' which clearly states the specific action and resource. It distinguishes this tool from siblings like start_lsp (initial startup) and get_diagnostics (inspection) by focusing on restarting an existing server.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists when to use the tool: 'if it becomes unresponsive, has stale data, or when you need to apply configuration changes' and 'when switching projects.' It provides clear context but does not explicitly mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_documentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses a key behavioral trait: 'The file remains open for continued analysis until explicitly closed.' This goes beyond the name and schema. However, it does not mention prerequisites like requiring the file to be opened first.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with action and use case. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage timing, and persistence behavior. Missing a clear statement of prerequisites (e.g., file must be opened first) and any error conditions, which would make it more complete for a no-output-schema tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with description 'Path to the file to save', and the tool description repeats this without adding syntactic or semantic detail. Thus the description adds no value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's action ('Save a file in the LSP server for analysis') and distinguishes it from sibling tools by tying it to the pre-analysis workflow. It explicitly mentions use before diagnostics/hover/completions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use context: 'Use this tool before performing operations...' It doesn't mention alternatives or when-not-to-use, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diagnosticsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing the default behavior when file_path is omitted (returns diagnostics for all open files) and the prerequisite that files be opened first. It doesn't describe the return format, but the read-only nature is implied by 'Get' and the description adds useful context beyond the bare tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences that front-load the main action and then provide optional behavior and prerequisites. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single optional parameter and no output schema, the description adequately covers purpose, usage context, and key behavioral nuance (all open files). It could explicitly mention the output structure, but for a diagnostics tool the expected return is generally understood.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool description adds no new meaning beyond the input schema: the schema already states that file_path is optional and that omitting it returns diagnostics for all open files. Since schema coverage is 100%, and the description merely repeats the parameter's behavior, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves diagnostic messages for files, using a specific verb ('Get') and identifying the resource (diagnostics for files). It distinguishes itself from sibling tools like get_completions or get_code_actions by focusing on errors, warnings, and language-server-detected issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this tool to identify problems in code files' and notes a prerequisite (files must be opened). It doesn't explicitly name exclusions or alternatives, but the clear scope provides sufficient guidance for when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_documentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It notes that 'The file remains open for continued analysis until explicitly closed,' which is a critical stateful behavior beyond what the schema reveals. It doesn't cover potential errors or return values, but the key persistence trait is disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and includes only necessary context about usage and persistence. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description is sufficient. It explains what, when, and the stateful behavior. The sibling tool list (e.g., close_document) provides additional context about the lifecycle, but the description alone already covers the essential aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the parameter 'file_path' described as 'Path to the file to open.' The description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Open') and resource ('file in the LSP server') with a clear purpose ('for analysis'). It distinguishes itself from siblings like close_document and get_diagnostics by stating it is a prerequisite for those operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'before performing operations like getting diagnostics, hover information, or completions for a file.' It doesn't explicitly mention when not to use or name alternatives, but the guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_log_levelA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses the effect on log output volume and its troubleshooting value. It does not cover persistence or broader side effects, but for a simple setter this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, then adding the level list and warning. Every sentence contributes value; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an enum and no output schema, the description fully covers the purpose, parameter semantics, and usage caveats. It is complete and self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema coverage is 100%, the description adds meaning by listing the levels in order of verbosity (emergency to debug), which clarifies the semantic ordering of the enum values and the practical impact of choosing a level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sets the server logging level and controls log verbosity. It uses a specific verb ('Set') and resource ('server logging level'), and is easily distinguished from siblings, which handle document/LSP operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool (to control verbosity and troubleshoot issues) and includes a caveat that increasing verbosity may generate large output. It does not explicitly mention alternatives, but no sibling tool serves the same purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_lspA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses that the tool must precede all other LSP calls and that file paths resolve relative to the root. It does not cover failure modes or return values, but the critical behavioral traits are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: what it does, when to call it, and how to set the parameter. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one param, no output schema), the description covers purpose, usage constraints, parameter semantics, and behavioral effects on other tools. It is complete for the intended use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only describes root_dir as 'The root directory for the LSP server'. The description enriches this with concrete examples (tsconfig.json, package.json) and explains path resolution implications, which is critical for correct usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Start' with a clear resource 'LSP server' and a required parameter 'root directory'. It clearly differentiates from sibling tools like get_completions or restart_lsp_server by establishing itself as the initialization step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states 'This tool must be called before using any other LSP functionality', providing an unambiguous usage condition. It also gives guidance on choosing the root directory with examples, making it clear when and how to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.