Server Quality Checklist

Tool Scores

• generate_migration_reportB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description lacks any behavioral traits such as whether the tool is read-only, requires specific permissions, or has side effects. It only describes the output content, not the operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a bullet list of report contents and a clear parameter list. It is front-loaded with the purpose, and every sentence provides value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, output schema exists), the description covers the report contents and parameter roles. It lacks behavioral context and prerequisites, but is otherwise complete for a report generation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaningful parameter explanations for most parameters (e.g., ingress_yaml, scheme, health check fields). Some parameters like gateway_grouping lack detailed meaning, but overall it compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it generates a comprehensive Markdown migration report and lists its contents. However, it does not explicitly differentiate from sibling tools like analyze_ingress or convert_to_gateway_api, which are more specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It implies a comprehensive use case but does not state when to prefer it or when to use sibling tools instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_ingressA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It states the tool analyzes and returns a summary but does not disclose whether it modifies anything, requires network access, or has rate limits. As an analysis tool, it is likely read-only, but this is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear structure: a brief purpose paragraph followed by parameter details. Every sentence adds value, and the information is front-loaded. No redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool's purpose and parameters are well-covered. The description lists output components, and an output schema exists (though not shown). Missing behavioral context (e.g., error handling, idempotency) but adequate for the complexity. Could mention it is a static analysis tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates excellently with an 'Args' bullet that explains each parameter's meaning, format (e.g., multi-document supported, file path), and optionality. This adds significant value beyond the schema's type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes Nginx Ingress configuration and returns a structured summary listing specific aspects (annotation compatibility, route type distribution, etc.). However, it does not differentiate from sibling tools like check_prerequisites or convert_to_gateway_api, missing an explicit statement of its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, workflow positioning, or exclusions. The sibling tool names offer context, but the description itself lacks any usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_prerequisitesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It only lists parameters and their meanings, but does not describe what happens when prerequisites fail, whether the tool makes any changes, or other side effects. This is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence for purpose followed by a parameter list. Every sentence adds value, and the structure is front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists (so return values are covered), the description lacks contextual completeness regarding behavior, side effects, or error handling. With no annotations, more detail is needed for a fully adequate description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description provides clear, meaningful explanations for all three parameters (lbc_version, needs_l4, tls_hostnames), including default behaviors like 'Leave empty to skip' and conditions for setting needs_l4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check if the target environment meets AWS Gateway API migration prerequisites.' It uses a specific verb ('Check') and resource ('prerequisites'), and distinguishes from sibling tools like convert_to_gateway_api.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage as a preliminary step for AWS Gateway API migration and explains parameter usage (e.g., 'Leave empty to skip' for lbc_version). However, it does not explicitly state when to use this tool versus alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_to_gateway_apiA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It outlines the pipeline steps but does not disclose error handling, idempotency, or whether the tool is destructive (likely it returns converted YAML without side effects).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a summary line, pipeline steps, and parameter list. It front-loads the purpose. While slightly lengthy, the parameter list is justified due to low schema coverage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (12 parameters, 0% schema coverage) and presence of output schema, the description covers the conversion pipeline and parameter meanings well. It lacks mention of error conditions or output format, but output schema may address that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description adds value by explaining most parameters, e.g., 'ingress_yaml: Nginx Ingress YAML content or file path' and 'gateway_grouping: Gateway consolidation strategy...'. However, some health-check parameters are only named without detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Convert Nginx Ingress YAML to AWS Gateway API resources (full pipeline)', with a specific verb and resource. It distinguishes from sibling tools like analyze_ingress and validate_output by being the actual conversion step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for converting Nginx Ingress to AWS Gateway API but does not explicitly state when to use vs alternatives like check_prerequisites or generate_migration_report. No when-not guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_outputA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It lists checks but does not disclose whether the tool is read-only, requires authentication, or has side effects. It does not describe the output format or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with the main purpose stated first, followed by a bullet list of checks. Every sentence adds value without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter and uses an output schema, the description covers the key aspects: what the tool does, what it checks, and the parameter's dual nature. It is sufficient for a single-parameter validation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds meaning by clarifying that yaml_content can be either actual YAML content or a file path. This goes beyond the schema's type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates Gateway API YAML against schema rules and AWS LBC constraints, listing specific checks like required fields, L4/L7 mixing, etc. This distinguishes it from siblings such as analyze_ingress or convert_to_gateway_api.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for validation but does not explicitly state when to use this tool versus alternatives. No exclusions or when-not-to-use guidance is provided; the sibling tools are not compared.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.