Server Quality Checklist

Tool Scores

• nvim_launchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds useful behavioral context: 'isolated, disposable', 'owned by this server', 'attach to it as a UI client', and 'returns a session id'. However, it doesn't disclose lifecycle details (e.g., what happens if nvim_close isn't called, cleanup behavior, or resource limits), leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, then the return value, then the cleanup instruction. Every sentence earns its place; no fluff or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and the input schema has 100% coverage, the description sufficiently contexts the tool as the starting point for a Neovim session and the counterpart to nvim_close. It could mention that other tools depend on this session id, but the phrase 'for the other tools' implies that clearly enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so per baseline the description doesn't need to explain parameters. The description adds no parameter-specific meaning beyond the schema, which is acceptable but earns no extra credit. It only mentions the session id return value, not any input params.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it launches an isolated, disposable Neovim instance, attaches to it as a UI client, and returns a session id. It uses a specific verb ('Launch') and resource ('Neovim instance'), and the 'isolated, disposable' detail distinguishes it from other nvim tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives clear usage context: this is the entry point that returns a session id for other tools, and explicitly says to use nvim_close when finished. It doesn't explicitly mention alternatives like nvim_attach, but the session id phrasing implies this tool is used first, which is adequate guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_observeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does well by specifying the output format (plain text rows with no cursor markers or line numbers), the reliability benefit (substring matching is reliable), and an important side effect (resetting the nvim_observe_diff baseline). However, it does not explicitly state that this is a read-only operation or any potential impacts on the Neovim session.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a concise three-sentence structure that front-loads the core purpose, then adds relevant details about output format and side effects. Every sentence provides necessary information without redundancy, making it a model of efficient prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no nested objects) and the presence of an output schema, the description sufficiently covers the tool's purpose, output characteristics, and side effect. It does not discuss error handling or performance, but these are not essential for a basic observe tool. The description complements the output schema by explaining the nature of the returned data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has full coverage (100%) for the single sessionId parameter, whose description is clear: 'Session id returned by nvim_launch or nvim_attach.' The description adds no further detail about the parameter, but the schema already provides sufficient meaning, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: returning the rendered Neovim screen as plain text rows plus metadata. It uses a specific verb ('Return') and specifies the resources (screen content, cursor, mode, size, window geometry). This distinctively separates it from sibling tools like nvim_observe_diff, nvim_input, and nvim_command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly suggests this tool is for reading the current screen state ("The screen is the source of truth"). It also notes the side effect of resetting the nvim_observe_diff baseline, which hints at a relationship to nvim_observe_diff, but it does not explicitly state when to use this tool over alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_commandA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It explains that rejected commands return errors with Neovim's own message and that it does not wait for the screen to settle, both important behavioral traits. This goes beyond basic semantics and provides useful operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and every word earns its place. It efficiently conveys purpose, error behavior, and synchronization semantics without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple interface with two well-documented parameters and an output schema, so the description does not need to explain return values. It covers error handling and the non-waiting behavior, which are key contextual details. Minor aspects like session validity or permission requirements are not mentioned, but they are implied by the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with detailed parameter descriptions, including examples for the command parameter and source of sessionId. The tool description adds no additional parameter-level details beyond what's in the schema, so it appropriately relies on the structured definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute an Ex command' with a specific verb and resource. It distinguishes from siblings by explicitly referencing nvim_input and contrasting the behavior, making it clear this tool is for Ex commands rather than key input or observation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when an Ex command needs to be executed. The mention of 'Like nvim_input' gives an implicit comparison, but it does not explicitly state when to prefer one over the other or mention alternatives for error handling. Still, the purpose is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_inputA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and discloses the asynchronous nature: 'Returns as soon as the keys are queued, not when the screen has updated.' It also warns about racing the redraw, which is valuable non-obvious behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action, and every word earns its place. The race-condition warning is essential and succinctly stated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully explains the tool's behavior and provides the necessary follow-up action (nvim_wait) for correct use. The presence of an output schema means return values need no explanation, and the description is complete for practical invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description need not repeat parameter details. The description does not add new semantic context beyond the schema's existing explanation of Neovim notation and session id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  'Send a key sequence to Neovim' is a specific verb+resource pairing that clearly states what the tool does. It distinguishes from siblings like nvim_command by emphasizing key sequence input rather than Ex commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit timing guidance: 'follow with nvim_wait before observing, otherwise the observation races the redraw.' It does not explicitly name alternatives, but the instruction to wait before observing is clear contextual usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_waitA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool blocks until a condition holds and that a timeout error includes the screen state for diagnosability. This covers key behavioral traits, though it does not explicitly state that it is read-only or mention other side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first front-loads the core purpose, and the second adds important error behavior. No wasted words and the structure is clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, the description need not explain return values. It provides enough context for when to use the tool (to avoid racing observations with redraws) and discloses timeout diagnostics. Combined with the rich schema, this is complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema already explains all parameters and their semantics. The description adds no specific parameter details beyond framing the 'screen condition' concept, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Block until') and resource ('a screen condition holds'), clearly stating the tool's function. It also differentiates itself from sibling tools by framing the wait as preventing observation from racing against a redraw, which is unique among the nvim_* tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool to synchronize with the screen before observing, avoiding race conditions. However, it does not explicitly name alternatives or state when not to use it, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_attachA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals a non-obvious side effect: 'Neovim sizes the screen to the smallest attached UI, so the requested size can shrink what the user sees.' It also explains ownership implications. However, it does not mention potential errors (e.g., invalid address) or what happens if already attached, so not a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the primary action, the ownership/termination caveat, and the screen-sizing consequence. No filler or redundancy, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the presence of an output schema, and the absence of annotations, the description covers the essential context: what it attaches to, the user ownership, and a critical side effect. It does not address error handling or prerequisites, but those are less critical given the schema and output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds behavioral meaning to the cols/rows parameters by explaining their impact on the user's screen, which goes beyond the schema's simple 'Requested screen width/height.' It does not fully compensate for all parameters but adds value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Attach as an additional UI to an already-running Neovim instance' with a specific verb and resource, distinguishing it from siblings like nvim_launch (launch new) and nvim_observe (observe). It precisely explains the tool's core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use the tool: for an already-running instance, and notes that nvim_close will refuse to terminate it because it belongs to the user. This implies it is for user-owned instances, but it does not explicitly name alternatives or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_closeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the termination action, the refusal behavior, and the rationale about user ownership. It could have mentioned potential side effects (e.g., unsaved changes) or irreversibility, but the given context is substantial and non-obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences. The main action is stated first, followed by the key exclusion. Every word earns its place; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description is complete: it states purpose, scope, exclusions, and rationale. It addresses the sibling tool context by explaining how this differs from attach-related operations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers sessionId 100% with a clear description, so the baseline is 3. The description adds behavioral meaning by noting that attach-mode session IDs are rejected, which is crucial semantic information for correctly using the parameter. This elevates the score above the schema-only baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Terminate') and clearly identifies the resource ('a Neovim instance this server launched'), immediately distinguishing it from sibling tools like nvim_launch and nvim_attach. It also explicitly contrasts with attach-mode sessions, leaving no ambiguity about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool (for sessions launched by the server) and when not to (refuses attach-mode sessions). It explains why attach sessions are excluded, providing a clear alternative (they are released on server shutdown), which is excellent guidance for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nvim_observe_diffA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses key behavioral traits: it returns before/after text for changed rows, includes deltas, and importantly notes 'Resets the baseline.' This is crucial for understanding the tool's stateful interaction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the core function, and includes the comparison and behavioral note without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter and an output schema exists, the description covers all necessary aspects: what the tool returns, how it differs from nvim_observe, and the baseline reset behavior. It is complete without needing extra detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, sessionId, is already fully described in the schema as 'Session id returned by nvim_launch or nvim_attach.' The tool description adds no extra meaning beyond that, and schema coverage is 100%, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns only changes since the last observation, listing specific data types (changed screen rows, cursor/mode/size deltas). It also explicitly distinguishes itself from sibling nvim_observe by presenting a smaller, targeted alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Much smaller than a full nvim_observe for verifying a single action.' This tells the agent when to choose this tool over its sibling and implies nvim_observe for full state needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.