Server Quality Checklist

Tool Scores

• get_completionsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a prerequisite (file must be opened) and implicitly signals read-only behavior via 'Get'. However, it does not detail side effects, error conditions, or requirements like a running LSP server, so behavioral transparency is only moderate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a purpose: purpose, content, and usage/prerequisite. It is front-loaded with the core action and remains free of fluff, though the phrase 'Helpful for code assistance' is somewhat generic.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four straightforward parameters and no output schema, the description covers the essential usage constraints: location specificity, context dependence, and the prerequisite of an open file. It doesn't explain return format, but the description's 'completion options' sufficiently informs what the response contains for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage with descriptions for all four parameters. The description adds context about what the completions include (variable names, etc.) but does not add new parameter-specific details beyond the schema. Per the rubric, this is a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Get') and resource ('completion suggestions'), explicitly scoping to a file location. It distinguishes from siblings like get_diagnostics and get_code_actions by emphasizing completion context (variable names, function calls, object properties). This leaves no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: when to retrieve completions, a concrete example (determining functions available in a package when changing libraries), and a prerequisite (file must be opened). It doesn't explicitly mention alternative tools, but the context is sufficient to guide an agent to select this tool over its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_info_on_locationA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses a key prerequisite ("Requires the file to be opened first") and implies a read-only operation via "retrieve." However, it does not mention error behavior, return format details, or any side effects, leaving notable gaps for a tool with no annotation safety net.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, with the core action in the first sentence. The subsequent sentences elaborate on use cases and the file-open prerequisite without significant redundancy. It is slightly longer than strictly necessary but every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no output schema, the description covers the essential aspects: what it does, what information it retrieves, when to use it, and a critical prerequisite. It does not explain failure modes for invalid locations or unopened files, but for a hover tool this is acceptable. The return content is described in broad terms, which suffices in the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all four parameters, so the baseline is 3. The description does not add extra meaning beyond the schema; it references "specific location" (mapping to line/column) but provides no additional syntax, relationships, or examples that would elevate the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: "Get information on a specific location in a file via LSP hover," clearly identifying the tool's function. It further distinguishes from siblings by detailing the nature of information (type info, documentation, contextual details), which differentiates it from get_completions or get_code_actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: "Use this whenever you need to get a better idea on what a particular function is doing in that context," and lists specific scenarios like understanding variable types, function signatures, and module documentation. It does not explicitly name alternatives or state when-not-to-use, so it misses the top tier, but the context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_code_actionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the prerequisite (file opened first) and gives examples, but does not state whether the tool has side effects, what it returns, or error behavior. This is partial transparency but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at three sentences, each serving a purpose: the first states the action, the second provides usage and examples, and the third notes the prerequisite. There is no fluff or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, and prerequisite, which is sufficient for a straightforward tool. However, since no output schema exists, it does not explicitly explain what the tool returns (e.g., a list of code action objects). This omission prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all parameters described. The description adds minimal parameter-related meaning beyond mapping 'specific range' to the line/column fields. It does not enhance the schema's parameter descriptions, meriting the baseline score for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get code actions for a specific range in a file.' It elaborates with examples of code actions (adding imports, fixing errors, implementing interfaces) and distinguishes itself from siblings like get_completions or get_diagnostics by focusing on refactorings and quick fixes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance ('Use this tool to obtain available refactorings...') and a key prerequisite ('Requires the file to be opened first.'). However, it does not explicitly name alternatives or when not to use this tool, so it falls short of a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diagnosticsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses two key behaviors: returning diagnostics for all open files when file_path is omitted, and requiring files to be opened first. However, it doesn't mention side effects (likely none), return format, or failure modes, which would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, with every sentence adding value: purpose, usage, and behavioral notes. It is front-loaded with the verb and resource, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 optional param, no output schema), the description covers purpose, usage, default behavior, and a prerequisite. It lacks explicit mention of the return structure, but for a diagnostics tool this is relatively clear from the context and typically sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the single parameter (file_path) with its optionality and default behavior, giving 100% coverage. The description repeats this information without adding extra semantic detail, so it meets the baseline for covered parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get diagnostic messages (errors, warnings) for files.' It uses a specific verb ('get') and resource ('diagnostic messages') and differentiates from sibling tools like get_completions and get_code_actions by focusing on error identification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent when to use it ('Use this tool to identify problems in code files') and provides important usage details: optional file_path behavior and the prerequisite that files must be opened first. It does not explicitly name alternatives or exclusions, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_documentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description shoulders the burden of behavioral disclosure. It does reveal a key behavioral trait—'The file remains open for continued analysis until explicitly closed'—which is useful. However, it omits other details such as error handling (e.g., if the file path does not exist) or whether reopening a file has any side effects, so the transparency is moderate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and every clause earns its place. It packs the core purpose, usage timing, and persistence behavior without redundancy or excessive detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description sufficiently covers the tool's role, use case, and lifecycle for a simple open operation. It does not detail return values, but there is no output schema and the tool likely returns nothing important. Given the simplicity and strong schema coverage, the only missing elements are edge-case behaviors, which do not heavily detract.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for both parameters (file_path and language_id), covering 100% of parameters. The tool description adds no extra parameter-level detail beyond their purpose in the overall workflow, so the baseline score of 3 is appropriate given the schema's thorough coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Open a file in the LSP server') and its purpose ('for analysis'). It distinguishes from sibling tools like close_document and get_diagnostics by framing it as a prerequisite step, making its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this tool before performing operations like getting diagnostics, hover information, or completions for a file,' providing direct guidance on when to invoke it. It also notes the file remains open until explicitly closed, implicitly suggesting the companion tool close_document for cleanup, though it does not name an explicit alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restart_lsp_serverA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly indicates a restart/reset behavior and optional reinitialization, but does not disclose potential side effects like losing document state or requiring reconfiguration. The description implies a reset but lacks explicit detail about consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the action 'Restart the LSP server process,' and uses three sentences to convey purpose, when to use, and optional behavior. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description adequately covers purpose, triggers, and the optional root directory behavior. It even provides context for troubleshooting and project switching, making it reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, as the parameter 'root_dir' has a clear description. The tool description adds 'Can optionally reinitialize with a new root directory,' which slightly reinforces the schema but does not add significant new semantics beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Restart the LSP server process' with a specific verb and resource. It distinguishes itself from sibling tools like start_lsp by focusing on restarting an existing process, and mentions use cases like unresponsiveness and stale data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'if it becomes unresponsive, has stale data, or when you need to apply configuration changes' and 'when switching projects.' It also notes the option to reinitialize with a new root directory, but does not explicitly mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_log_levelA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden of behavioral disclosure. It explains the effect on log verbosity and the trade-off of high verbosity, but it does not mention persistence, whether the change applies immediately, or what the response looks like. Some behavioral gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the purpose, and every sentence provides useful information. No fluff or repetition. The level ordering and troubleshooting note are essential and efficiently integrated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter setter, the description covers the purpose, parameter semantics, and usage guidance. It does not describe the return value or whether the setting persists, but given the tool's simplicity and the schema's completeness, these gaps are minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already covers the parameter fully with an enum and description, the description adds semantic value by ordering the levels from least to most verbose and explaining the practical implication of choosing a higher level. This goes beyond the schema's bare listing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Set the server logging level,' which is a specific verb and resource. It further clarifies the scope as the LSP MCP server, distinguishing it from sibling tools like get_diagnostics or restart_lsp_server. The purpose is unambiguous and concise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this tool to control log verbosity, and notes that increasing verbosity can help troubleshoot issues but may produce large output. It does not explicitly name alternatives or exclusions, but the tool is unique among siblings, so the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_documentA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden. It discloses behavioral effects beyond simply closing—freeing resources and reducing memory usage—but doesn't detail edge cases like closing a non-open file or error behavior. This is decent but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the action, followed by practical advice. Each sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one parameter and no output schema, the description adequately covers the purpose and usage. It lacks explicit mention of return values or failure modes, but these are less critical given the simple nature.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes file_path with a clear description, and the tool description adds no additional parameter semantics. Baseline for high schema coverage applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Close') and resource ('a file in the LSP server'), distinguishing it from sibling tools like open_document and get_diagnostics. It is specific and not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use the tool: when done with a file, to free resources, and in long-running sessions or large codebases. This gives clear context, though it doesn't explicitly contrast with alternatives beyond the implied inverse of open_document.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_lspA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the ordering dependency and explains that file paths are resolved relative to the root, which is important behavioral context. It does not cover edge cases like double-start or return values, but the provided details are meaningful for a start operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no fluff: the first states the action, the second explains the prerequisite, and the third clarifies path resolution. Each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter start tool, the description is fairly complete, covering purpose, prerequisite, and parameter semantics. It lacks details on return behavior or error conditions, but the tool's simplicity and the emphasis on its role as a prerequisite mitigate that gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema covers 100% of the parameter, the description adds valuable semantics by explaining what the root directory should contain (config files like tsconfig.json, package.json) and how it is used for path resolution. This goes beyond the schema's one-line description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start the LSP server') and the key input (root directory). It distinguishes itself from the sibling restart_lsp_server by emphasizing 'must be called before using any other LSP functionality', positioning it as the initialization step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly provides a strong when-to-use instruction: 'This tool must be called before using any other LSP functionality.' It also gives context on root directory selection. However, it does not explicitly mention alternatives or when not to use it, though restart_lsp_server is an implicit alternative for already-running scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.