Server Quality Checklist

Tool Scores

• assess_retrofit_levelA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains the tool's action (recommend with reasoning) but does not detail potential side effects, error behavior, or whether it modifies anything. Given the read-only nature implied, this is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the purpose, and contains no extraneous information. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the tool has no output schema and no annotations, the description adequately covers the input requirements and purpose. However, it lacks detail on the output format or example reasoning, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes both parameters clearly, including the mutual exclusivity. The description adds minimal new information beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'recommend' and the resource 'retrofit level', distinguishing it from sibling tools 'audit_intent_spec' and 'generate_intent_spec_scaffold' which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies that either skill_text or file_path must be provided (not both) and grounds the reasoning in a framework. However, it does not explicitly state when to use this tool versus siblings or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_intent_specA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It discloses pagination behavior and that the tool performs an audit (implying read-only), but does not mention other behavioral traits like required permissions, rate limits, or error handling. The transparency is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences, each adding unique value: purpose, input options, and pagination. It is front-loaded and contains no unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains input and pagination but does not describe the output format or what the audit results look like. Since there is no output schema, this is a gap. It is moderately complete for a tool with simple inputs but lacks output details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context beyond the schema by explaining the pagination mechanism and that the response includes a next_chunk_token. This helps the agent understand how to handle long inputs effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose: auditing an intent spec against a specific checklist and anti-patterns. It uses a specific verb ('audit') and resource ('intent spec'), and implicitly differentiates from sibling tools like generate_intent_spec_scaffold and assess_retrofit_level.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to provide input (spec_text or file_path) and pagination, but does not explicitly state when to use this tool versus siblings or provide exclusion criteria. The guidance is clear for parameter usage but lacks tool selection context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_intent_spec_scaffoldA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description states tool returns a template but does not explicitly confirm read-only behavior or discuss limits. Provides useful context on what gets pre-filled. Adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. Purpose and usage clearly stated. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema; description does not detail the format or structure of the returned template (e.g., YAML, sections). Users may need to know what 'full-9-section' contains. Parameter coverage is good, but completeness could be improved by describing output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover all 4 parameters clearly. The tool description adds context by stating they 'pre-fill' the template fields, linking parameters to output. Since schema does most of the work, score is slightly above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states tool returns intent-spec templates, lists three variants ('blank', 'level-1-mvr', 'full-9-section'), and notes usage for new or retrofit scenarios. Distinguishes from siblings: audit_intent_spec and assess_retrofit_level are different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: 'Use this when starting a new agent/skill or retrofitting an existing one.' Does not explicitly state when not to use, but siblings make distinction clear. Could be improved by adding exclusions (e.g., 'not for auditing').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.