Server Quality Checklist

Tool Scores

• set_project_rootA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description only mentions the prerequisite nature but omits behavior on invalid paths, side effects, or persistence, which is insufficient for a configuration tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, no wasted words, front-loads essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose and prerequisite but lacks return value or error details; still adequate for a simple single-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameter details; description adds prerequisite context but no extra parameter semantics, meeting baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sets the project root directory and must be used before other tools, distinguishing it from sibling analysis/test tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states it must be called before other tools, providing clear usage context, though it lacks when-not or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_testsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses recursive directory search, pattern filtering, and output of structured file info. Since no annotations are provided, the description carries the full burden, and it adequately conveys read-only exploration behavior without side effects, though it could explicitly state no modifications occur.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement and a separate usage section. It is concise but not overly terse; every sentence adds value, though the 'USE WHEN' repetition slightly reduces tightness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 optional param, no output schema), the description covers purpose, usage criteria, patterns, and prerequisites. However, it lacks explicit output format details and error handling, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one optional parameter 'path' with a comprehensive description covering relative/absolute paths and default behavior. The tool description adds no additional semantic value beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool discovers and catalogs test files across the project, supporting patterns like .test.* and .spec.*. It distinguishes from sibling tools like run_tests and analyze_coverage by focusing on exploration rather than execution or analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes explicit 'USE WHEN' guidance, listing specific user queries like 'what tests exist' and contexts. It also states a prerequisite ('Requires set_project_root to be called first'), which helps prevent misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_coverageA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses automatic exclusion of non-production files, prevention of analysis on test files, and configuration via vitest.config.ts. No annotations exist, so description carries full burden; it adequately informs about safety and no destructive actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with key information upfront and a dedicated 'USE WHEN' section. Slightly verbose but every sentence adds value; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description explains what metrics are returned (lines, functions, branches, statements) and provides example patterns for exclude parameter. Covers prerequisites, automatic behaviors, and usage triggers comprehensively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good descriptions. Description adds context beyond schema: target should be source code (not test files), relative paths from project root, and why target is required (prevents slow full analysis).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it performs test coverage analysis with metrics for lines, functions, branches, and statements, and distinguishes from sibling tools (list_tests, run_tests, set_project_root) by focusing on coverage rather than test execution or listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly lists when to use (coverage-related queries) and states prerequisite 'Requires set_project_root to be called first'. Also recommends preferring this over raw vitest coverage commands. Missing explicit when-not usage, but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_testsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses safety guards against full project runs, intelligent format detection, console log capture, and monorepo support. Missing details on side effects or return structure, but sufficient for its use case.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with sections, but slightly lengthy. It front-loads the main purpose and features. Could be trimmed slightly without losing clarity, but remains efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no output schema, and reasonable complexity, the description covers essential behavior. It mentions output is AI-optimized JSON but lacks explicit structure. Still, it provides enough context for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds value by explaining smart defaults for format, monorepo context for project, and debugging use for showLogs. It clarifies the required nature of target beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute Vitest tests' with specific features like AI-optimized output, format detection, log capture, and safety guards. It distinguishes itself from siblings such as analyze_coverage and list_tests by being the only test execution tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'USE WHEN' section listing scenarios like running tests, checking pass/fail, debugging failures, and hints like 'vitest-mcp:'. It also specifies prerequisite 'set_project_root' and advises preferring this tool over raw commands.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.