Server Quality Checklist

Tool Scores

• stale_scanA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so effectively. It explicitly states side effects: 'reads all markdown files and source files reachable from the project root (respecting .gitignore). Never writes, auto-fixes, or moves files.' It also describes the exit behavior: 'Exit 1 if any drift is found.' The only minor gap is that it doesn't mention performance characteristics or rate limits, but for a local file scanning tool, this is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose. Each sentence adds distinct value: purpose statement, usage guidelines, behavioral transparency, and return format/exit behavior. There's no wasted text, and the information is organized logically from general to specific.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description provides excellent contextual completeness. It covers purpose, usage guidelines, behavioral characteristics (including what it does and doesn't do), and output format details. The description compensates well for the lack of structured metadata by being comprehensive yet concise.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds some context about the 'git' parameter ('Set `git: true` to additionally flag docs...'), but doesn't provide additional semantic meaning beyond what's in the schema descriptions. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('detect', 'find') and resources ('documentation drift', 'README.md, CHANGELOG.md, and other project markdown'). It distinguishes itself from potential siblings by explicitly stating it's a 'pure reporting tool' that 'never writes, auto-fixes, or moves files', which differentiates it from tools that might perform automated fixes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool: 'before tagging a release, after large refactors or renames, when onboarding a new contributor, or as a periodic health check.' It also includes a specific conditional usage tip: 'Set `git: true` to additionally flag docs that have not been touched since a closely related source file changed significantly.' This gives clear context for both primary and advanced use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aware_syncA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so effectively. It details side effects (reads manifest files, writes/overwrites specific AI context files), explicitly states what it does not do ('Never modifies source code'), and describes the return behavior (plain-text summary, exit codes). It could improve by mentioning potential performance impact or file permission requirements, but covers core behavioral traits well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with clear sections: purpose statement, usage guidelines, side effects, and return behavior. Every sentence adds essential information with zero waste. It's appropriately sized for a tool with multiple behaviors and parameters, and front-loads the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, no annotations, and no output schema, the description provides comprehensive context about behavior, side effects, and returns. It covers what the tool does, when to use it, what files it interacts with, and the output format. It could slightly improve by explicitly mentioning error handling details or dependencies, but is largely complete for the given complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description adds value by explaining the practical implications of the 'dryRun' parameter ('report the files that would be written without touching disk') and suggesting usage ('Use this to preview changes before committing'), which goes beyond the schema's technical definition. It doesn't elaborate on 'path' parameter semantics, but the schema coverage is already complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('detect', 'regenerate') and resources ('tech stack', 'AI context files'), and distinguishes it from siblings by focusing on configuration file generation rather than validation, scanning, or task management. It explicitly names the files involved (CLAUDE.md, .cursorrules, etc.) and the configuration source (.aware.json).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use ('after adding or removing a framework/language', 'when AI context files fall out of date', 'when onboarding a new agent') and when not to use ('Do not call on every turn — run once per session or after stack changes'). It clearly differentiates this from sibling tools by its specific use case of maintaining AI context files based on tech stack detection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• velocity_end_taskA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so comprehensively. It details side effects (updates task row, shells out to git commands, computes embeddings, records calibration residuals, seals plans), execution constraints (5s timeout, safely no-ops outside git repo), and conditional behaviors based on status. This provides rich behavioral context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage guidelines, side effects, return values) and front-loads the core functionality. While comprehensive, some sentences could be more concise (e.g., the git command explanations are detailed but necessary). Overall, most content earns its place by providing essential behavioral context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex mutation tool with 11 parameters and no annotations or output schema, the description provides exceptional completeness. It covers purpose, usage guidelines, side effects, execution details, conditional behaviors, and return format. The detailed explanation of what happens for different status values and the comprehensive return value description compensate for the lack of structured output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 11 parameters thoroughly. The description doesn't add significant parameter-specific information beyond what's in the schema descriptions. It mentions some parameters indirectly (like status affecting embeddings), but doesn't provide additional syntax, format, or usage details for individual parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose with specific verbs ('Stop a task timer', 'record the outcome', 'return the actual duration') and clearly distinguishes it from its sibling 'velocity_start_task'. It identifies the exact resource being operated on (task timer started with velocity_start_task) and the comprehensive actions taken.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('immediately after finishing — or abandoning — any task started with velocity_start_task') and when not to skip it ('Always call, even on failed or abandoned outcomes; skipping leaves orphaned active rows'). It clearly references the alternative/sibling tool (velocity_start_task) and explains the consequences of misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• berth_checkA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so comprehensively. It details side effects ('enumerates listening TCP ports... reads only the config files... never opens, closes, or binds any ports'), safety aspects ('read-only and side-effect free'), and operational specifics like platform-dependent commands and exit codes, going well beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. Each sentence adds necessary information (usage guidelines, side effects, returns), but it could be slightly more streamlined by integrating some details more tightly, though no content is wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the tool (port scanning with multiple config sources and platform variations), no annotations, and no output schema, the description is highly complete. It covers purpose, usage, behavior, side effects, return values, and exit codes, providing all essential context for an AI agent to understand and invoke the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description adds value by implicitly contextualizing the 'path' parameter as the 'project root to scan' and the 'format' parameter's purpose in the 'Returns' section, but it doesn't provide additional syntax or format details beyond what the schema already documents, warranting a score above baseline but not the highest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Scan a project for port conflicts') and resources ('ports declared in common dev-server config sources'), distinguishing it from sibling tools like 'aware_sync' or 'envalid_validate' which likely perform different functions. It explicitly identifies what it does without being tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool ('before running `npm run dev`, `docker compose up`, or any other dev-server command — especially after switching branches in a monorepo'), including specific scenarios and timing. It also mentions it's 'Safe to call repeatedly,' which helps differentiate usage patterns from other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• envalid_validateA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and excels. It details side effects: 'reads .env, .env.local, and .env.{environment} from the project root, and reads the envalid schema.' It explicitly states safety: 'Does not write or transmit env values anywhere — validation is local-only.' It describes output behavior: 'Returns: plain-text, JSON, or markdown report... Exit 1 on any validation failure.' This covers critical behavioral traits beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by usage guidelines, side effects, and return behavior. Every sentence adds value: the first defines the tool, the second specifies use cases, the third details side effects and safety, and the fourth describes output and exit behavior. There is no wasted text, and information is presented in a logical flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (validation with side effects and conditional behavior), no annotations, and no output schema, the description provides comprehensive context. It covers purpose, usage, behavioral transparency (including safety and output), and parameter implications. The absence of an output schema is compensated by detailing the return format and exit behavior, making it complete enough for an agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema: it explains the purpose of the 'environment' parameter ('Controls which .env.{environment} file is loaded and which conditional schema rules apply') and implies usage of 'path' and 'format' through examples like 'project root' and output formats. However, it doesn't provide additional syntax or constraints for parameters beyond what the schema already documents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('validate .env files against envalid schema') and resource ('.env files'), with explicit differentiation from siblings by detailing what it catches: missing required variables, type mismatches, and values outside allowed ranges. This goes beyond a generic 'validate' statement to specify the exact validation scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'before booting the app locally, during CI, after a teammate adds a new required env var, or when switching between development and production configs.' It also includes a specific instruction to 'Pass `environment` to validate a specific .env.{env} file,' offering clear context for parameter usage without needing to reference alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• velocity_start_taskA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and excels at disclosing behavioral traits. It describes side effects ('inserts a new row into the local SQLite database'), computational behavior ('Computes a best-effort duration prediction by querying historical rows'), caching behavior ('predictions run locally and are cached per-task'), and privacy/configuration details ('Federated upload is disabled unless the user has explicitly opted in').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It begins with the core purpose, then provides usage guidelines, side effects, and return values in logical sections. Every sentence adds value: the first explains what the tool does, the second provides usage context, the third details side effects and computational behavior, and the fourth specifies return values.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 10 parameters, no annotations, and no output schema, the description provides excellent completeness. It explains the tool's purpose, when to use it, behavioral characteristics, side effects, computational approach, privacy considerations, and detailed return structure. The description compensates fully for the lack of structured metadata about outputs and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the overall purpose of parameters ('return a duration estimate derived from similar past tasks') and providing context about how parameters like category and tags affect historical matching. However, it doesn't provide specific guidance on parameter interactions or advanced usage patterns beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Start a timer', 'return a duration estimate') and resources ('discrete coding task', 'historical data'). It distinguishes from sibling 'velocity_end_task' by explaining this starts tasks while the other ends them, and from other siblings by focusing on time tracking with predictions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'before starting any distinct unit of work' with examples (bug fix, feature, refactor), advises 'one task per logical unit', warns against batching unrelated changes, and explicitly states to 'Always pair with `velocity_end_task`'. It also mentions when not to use (when federated upload is disabled unless opted in).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vow_scanA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so comprehensively. It clearly states side effects ('reads supported lockfiles... plus local node_modules / vendor manifests'), declares it's 'Read-only; no network requests', specifies exit codes ('Exit 1 if any dependency violates the policy or has an unknown license, 0 otherwise'), and documents platform limitations ('Pnpm, yarn, and go are not yet supported — vow exits with a clear error').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and efficiently organized. It begins with the core purpose, follows with usage guidelines, then details behavioral aspects, and concludes with return values. Every sentence serves a distinct purpose without redundancy, making it easy to parse while providing comprehensive information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description provides exceptional completeness. It covers purpose, usage scenarios, behavioral characteristics (including side effects, limitations, and exit codes), parameter semantics, and return format details. The description fully compensates for the lack of structured metadata, making the tool's functionality and constraints completely understandable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While schema description coverage is 100%, the description adds valuable semantic context beyond the schema. It explains the practical implications of the 'production' parameter ('skip devDependencies and audit only what actually ships') and provides usage guidance ('Use this for release-gate checks; leave false for full audits'). However, it doesn't add significant meaning for the 'path' and 'format' parameters beyond what the schema already documents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('scan all dependency licenses', 'validate each package against policy', 'flagging disallowed licenses') and distinguishes it from siblings by focusing on license compliance scanning. It explicitly identifies the resource (dependency licenses in a project) and the action (scanning and policy validation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool: 'before shipping a release, when adding a new dependency, during compliance or legal review, or as a CI gate.' It also offers specific parameter guidance ('Set `production: true` to skip devDependencies') and mentions limitations ('Pnpm, yarn, and go are not yet supported'), giving clear context for appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.