Server Quality Checklist

Tool Scores

• respec_validateC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It discloses the use of Puppeteer for rendering and that it does not write output. However, it omits side effects, permissions needed, or potential failures. The disclosure is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short and easy to read, but it sacrifices essential detail. While concise, it does not fully earn its place by covering key aspects beyond purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given five parameters, no output schema, and no annotations, the description is incomplete. It does not explain output format, error handling, prerequisites, or any usage context beyond basic validation. More detail is needed for adequate completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description provides no explanation for any of the five parameters (profile, status, overrides, source, output). No defaults or formats are mentioned. The description fails entirely to add meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates a ReSpec document, runs rendering via Puppeteer, and returns diagnostics. The title adds 'full render, no write', distinguishing it from write tools. However, it could be more specific about the types of validation performed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Does not write output', hinting at a contrast with respec_build, but provides no explicit guidance on when to use this tool over siblings like respec_preflight or respec_scaffold. No criteria for preferred usage are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• respec_buildC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions rendering, writing output, and returning compliance results, but lacks details on side effects (e.g., file overwriting), idempotency, permissions, or error behavior. Critical behavioral traits for a build tool are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and to the point. It front-loads the main action. Minor improvement could be breaking into two sentences for readability, but it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters (none required), no output schema, and no parameter descriptions, the description is incomplete. It does not explain parameter roles, compliance result format, or prerequisites, leaving significant gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not elaborate on any of the five parameters (profile, status, overrides, source, output). It mentions 'source file' and 'static HTML' but does not link them to parameters, providing no added value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool renders a ReSpec source file to static HTML using a repo-local profile, writes output, and returns compliance results. It uses a specific verb ('renders') and resource ('ReSpec source file'), and distinguishes itself from sibling tools like respec_list_profiles (listing) and respec_validate (validation) by focusing on the build action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings. It does not mention prerequisites, when-not scenarios, or alternative tools. The description only states what the tool does, not the context for its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• respec_scaffoldC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It mentions a path constraint but does not disclose side effects (e.g., overwriting behavior), permissions needed, or what happens on failure. The behavior beyond creation is opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, no wasted words. The first sentence defines purpose, the second adds a key constraint. It is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no annotations, the description should cover return values, parameter interactions (e.g., how 'overrides' is used), and typical usage scenarios. It lacks this context, making it incomplete for an agent to use reliably.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'repo-local profile' and 'status-specific template', hinting at the 'profile' and 'status' parameters, but leaves 'overrides', 'source', and 'output' unexplained. This is insufficient for 5 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Creates') and the resource ('ReSpec source document'), and specifies the inputs ('repo-local profile and status-specific template'). It also distinguishes from sibling tools: respec_build is for building, respec_list_profiles for listing, etc., so this tool's role is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like respec_build, respec_validate, or respec_preflight. It does not state prerequisites or conditions where scaffolding is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• respec_preflightA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It discloses it is fast, source-only, and no rendering, plus the checks performed. However, it does not mention side effects, destruction, or rate limits. With zero annotations, a 3 is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose and usage. Every sentence adds value with no fluff. Excellent conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no parameter descriptions, the description is incomplete. It does not explain how to use the source parameter, what output produces, or the return format. Only the high-level policy checks are mentioned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the 5 parameters (profile, status, overrides, source, output). The description only lists what the tool checks, adding no parameter-specific meaning. This is a critical gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fast policy checks against the source file without rendering', listing specific checks like required sections and forbidden phrases. It distinguishes from siblings like respec_build which does rendering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear when-to-use guidance: 'Use this before edits to catch policy violations cheaply.' It implies that for rendering one should use respec_build, though it does not explicitly name alternatives or when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• respec_list_profilesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It states the tool lists profiles from specific file sources, which implies a read-only operation. While it does not explicitly say it does not modify data, the verb 'list' and the context make the behavior clear. Additional context about the return format or side effects would improve it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the tool's purpose and data sources. It is front-loaded with the action and resource, with no superfluous information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no complexity), the description provides the essential information: the sources of the profiles. It does not mention the return format or that the list may be empty, but for a list operation this is sufficient. A 5 would require more detail about the output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is empty with 100% coverage. According to the guidelines, a baseline score of 4 is appropriate since the description cannot add meaning beyond the schema. The description does not need to elaborate on parameters, so this score reflects the minimum viable for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose: listing repo-local ReSpec MCP profiles from specific sources (respec-mcp.config.json and respec-mcp/profiles/*.json). It uses a specific verb-resource combination that distinguishes it from the sibling tools (build, preflight, scaffold, validate), which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to list profiles, and given the sibling tools are all for different operations (building, preflighting, scaffolding, validating), there is no ambiguity. However, it does not explicitly state when to use this tool over others or provide any exclusion criteria, but the context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.