Server Quality Checklist

Tool Scores

• vitest_scaffold_configB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and partially succeeds: it discloses the core effects—creating two files and adding a script—and 'deterministically' signals predictable output. However, it does not state whether existing files are overwritten, whether package.json must already exist, or other side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, dense sentence that front-loads the action and lists concrete outputs. No filler, no repetition, every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutating scaffold tool with no output schema and no annotations, the description is thin. It omits lifecycle context (e.g., run after vitest_classify_package), behavior on pre-existing files, and what a successful invocation returns or changes beyond the stated artifacts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both packagePath and tier described inline. The description adds no parameter-specific detail, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('generates') and names the exact artifacts: vitest.config.browser.mjs, test/setup.ts, and the package.json test:vitest:browser script. This makes the tool's function unambiguous, though it does not explicitly differentiate from sibling tools like vitest_apply_ast_codemods.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus its siblings. There is no mention of prerequisites, pipeline ordering, or circumstances where a different tool (e.g., vitest_apply_ast_codemods) should be used instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vitest_classify_packageB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of disclosing behavior. It implies a read-only analysis but does not explicitly state side effects, prerequisites, or the exact form of the classification result. The listed tier categories are helpful but not a full behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler. The parenthetical list of tier categories adds useful specificity while keeping the description compact and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the purpose and gives example output categories, which is adequate for a simple one-parameter classifier. However, since there is no output schema and no annotations, an agent may still need to infer exact tier strings, return format, and any repository-side requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single param, so the schema already documents packagePath as an absolute path. The description adds only the context that it refers to a repository package, matching the schema without adding meaningful new semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('Analyzes') and a specific resource ('a package in the repository'), and specifies the output ('Vitest migration tier') with concrete example categories. This clearly distinguishes it from sibling tools like vitest_scaffold_config or vitest_run_verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus siblings such as vitest_scaffold_config or vitest_lookup_knowledge_bank. It never states whether classification should precede scaffolding, verification, or any other workflow step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vitest_run_verificationB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully explain what happens when the tool is invoked. It only says it 'runs test verification', not whether it executes commands, modifies the package, creates artifacts, or what the verification result looks like. The supported targets and legacy label add some context, but the actual behavioral impact is mostly undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core action and includes the key discriminator (supported test runners). It wastes little space, though 'in the target package' is partly redundent with the packagePath parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple and both parameters are fully documented in the schema. However, there is no output schema and the description does not explain what the tool returns or what success looks like, which an agent would need to verify result correctly. This is an acceptable baseline but leaves notable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents both parameters at 100% coverage, so the description is not needed for basic semantics. However, it adds value by grouping the enum values into meaningful categories: Vitest Browser vs. Vitest Node, and legacy Moocha Node / Karma Browser. This helps an agent understand the target parameter beyond raw enum strings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('runs') and a specific resource ('test verification') and details the exact supported runners: Vitest Browser, Vitest Node, Mocha Node, Karma Browser. This clearly differentiates it from siblings like vitest_classify_package or vitest_scaffold_config, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to call this tool instead of or after the sibling tools. There is no mention of prerequisites, when verification is appropriate, or how to choose among the four targets. The target enum gives options but no decision-making context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vitest_apply_ast_codemodsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden of disclosing side effects. It says 'Executes TypeScript AST transformations' but never explicitly states that files may be modified in place, whether changes are reversible, what backups or guards exist, or failure behavior. For a mutation-style tool, this is a meaningful transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence with no filler. The verb, mechanism, and target outcomes are front-loaded, and every clause contributes information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers what the tool does and the required input, but with no output schema and no annotations it omits return behavior and the fact that AST transformations typically rewrite source files. For a one-parameter tool the gaps are moderate, but they are still meaningful for an agent deciding whether to apply it safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, packagePath, is fully documented in the schema as 'Absolute path to the package directory,' which already gives the agent the necessary meaning. The description adds no additional param-specific detail, matching the baseline for 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific operation ('Executes TypeScript AST transformations'), the mechanism ('ts-morph'), and the concrete problem classes it fixes (isolated ESM type exports, Mocha timeouts, require() calls). This clearly distinguishes it from sibling tools that classify, scaffold, verify, or manage knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use context: apply it when a package needs automated AST fixes for the listed TypeScript/ESM/Mocha/CommonJS issues. It does not explicitly state when not to use it or name alternative codemod tools, so it stops short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vitest_list_knowledge_rulesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations and no output schema, the description carries the burden of behavioral disclosure. It clearly indicates an enumeration operation and notes that the set is 'currently known,' but it does not describe the return format, ordering, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no filler. It front-loads the action and resource and earns every word.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool, the description gives sufficient context: what is returned and where it comes from. It is slightly lacking in that it does not mention output shape or relationship to lookup_knowledge_bank, but this is a minor gap for such a simple operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to explain beyond what the schema already communicates. Baseline 4 is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Lists'), a clear resource ('all currently known migration rules'), and the source ('Knowledge Bank'). It is immediately clear what the tool does, though it does not explicitly differentiate itself from the sibling lookup tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Lists all currently known migration rules' implies this tool is for obtaining an exhaustive inventory of rules. However, it gives no explicit guidance about when to prefer this over the sibling vitest_lookup_knowledge_bank, nor any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vitest_lookup_knowledge_bankA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. 'Queries' does imply a non-destructive read operation, and the returned content is enumerated. However, it does not explain behavior for unknown signatures, response format, or whether any side effects such as logging or learning recording occur.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence that front-loads the action and resource, then states the value returned. There is no filler, repetition, or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter lookup tool, the description covers what the tool does, what it operates on, and what outputs to expect. There is no output schema, so the explicit mention of diagnosis, root cause, and fix template is valuable. The only notable gap is no-match behavior, but that is not essential for selecting or invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the only parameter, errorMessage, as the failing test error message or stack trace, so schema coverage is 100%. The description adds the idea of 'known error signatures' but provides no additional parameter semantics beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Queries'), a specific resource ('the permanent Knowledge Bank'), and an explicit output ('diagnosis, root cause, and fix template'). It is easy to distinguish from sibling tools like vitest_record_learning or vitest_list_knowledge_rules because it focuses on looking up known signatures rather than recording or listing all rules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended usage is implied: when you have a failing error message or stack trace and want a known diagnosis, root cause, and fix template. However, the description does not explicitly state when to prefer this over siblings such as vitest_list_knowledge_rules, nor does it mention any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vitest_record_learningA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose a key behavioral trait: the write is 'permanent' and goes into a Knowledge Bank. But it does not clarify what happens on duplicate signatures, whether existing entries are overwritten, or what the response/return behavior is.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with the action and resource. Every word contributes: 'newly resolved' indicates timing, 'permanent' signals persistence, and 'Knowledge Bank' names the storage context. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple record operation and the schema covers all parameters. But with no output schema and no annotations, the agent is left without important context like duplicate handling, return value, or whether prior lookup is recommended before recording. Sibling workflow context is hinted at but not explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are already well-documented. The description adds only high-level mapping ('error pattern, root cause, fix template') to the schema fields, without adding new semantic detail about the optional 'name' or 'commentTemplate' parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Records') with a concrete resource ('newly resolved error pattern, root cause, and fix template') and destination ('permanent Knowledge Bank'). It clearly distinguishes this write/learning tool from the read-oriented sibling tools like vitest_lookup_knowledge_bank and vitest_list_knowledge_rules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'newly resolved error pattern' gives clear context for when to use the tool: after encountering and resolving an error that should be persisted. However, it does not explicitly state when not to use it or route to alternative tools such as lookup or list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.