Server Quality Checklist

Tool Scores

• generate_docsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavior. It only mentions file generation without addressing side effects (e.g., overwriting), required permissions, or behavior on invalid input. The mutation aspect is implied but not detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence efficiently conveys core function without extraneous information. However, lacks structure (e.g., separate sections for usage) that could improve scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given moderate complexity (nested object parameter, no output schema), the description is insufficient. It does not explain the expected format of the 'design' object, what happens upon success or failure, or whether the output files are returned or written to disk. More context is needed for reliable agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of parameters with descriptions, so baseline is 3. The tool description adds no additional meaning beyond the schema, maintaining adequacy without enhancement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool generates ARCHITECTURE.md and .ai-context.yaml files based on architecture design results. It distinguishes itself from sibling tools by specifying its output format and target audience (human vs AI).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives, such as the obvious prerequisite of having a design from design_architecture. The description only states what it does, not the context of usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• design_architectureB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must reveal behavioral traits. It only states outputs are generated, but does not disclose whether files are overwritten, required permissions, rate limits, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences clearly state purpose and outputs in a front-loaded structure. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 11 parameters and no output schema, the description is ambiguous: it mentions 'interactive questions' but the tool expects all parameters upfront. It lacks details on return format or workflow, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with individual parameter descriptions. The description does not add additional meaning beyond summarizing the parameter groups, so it meets the baseline for high coverage without enhancement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool designs architecture and generates specific files (ARCHITECTURE.md and .ai-context.yaml). It clearly distinguishes from sibling tools like estimate_cost or generate_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when designing architecture by analyzing platform, scale, budget, but does not provide explicit when-not-to-use guidance or comparisons with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_costB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description only states basic purpose. No disclosure of side effects, authentication needs, error handling, or whether it's read-only. For a cost estimation tool, agents need to know if it uses live pricing or static data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded, no redundancy. Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks details about output format, supported regions, currency, or whether it makes API calls. For a cost estimator with nested parameters and no output schema, description is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters. Description adds overarching context (monthly cost, breakdown) but does not add detail beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb (calculate), resource (monthly estimated cost), and scope (architecture configuration, breakdown by service). Distinguishes from siblings like design_architecture and recommend_cpu_arch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. Alternatives are not mentioned. Usage is implied by the tool's clear purpose, but lacking explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scaling_strategyB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only states the tool suggests and provides, but does not disclose read-only nature, output format, or any side effects. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no waste, front-loaded with purpose. Efficient communication.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters and no output schema, the description is too brief. It does not explain output format, how the strategy is presented, or any prerequisites. Lacks detail for full understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add significant meaning beyond the schema's parameter descriptions. It mentions 'current architecture' and 'traffic increase scenarios' but that is already implied.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it suggests a scaling strategy for the current architecture and provides response plans for traffic increases. This is specific and distinct from siblings like design_architecture or estimate_cost.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for scaling strategy but does not explicitly state when to use it vs alternatives or when not to use it. No comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_cpu_archB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions cost and performance considerations but fails to disclose what the tool returns (e.g., single choice, comparison), whether it requires additional inputs, or any other behavioral traits. This minimal disclosure is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, no wasted words, and immediately front-loaded with the tool's core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should hint at the return format or additional details. It only says 'recommends' but doesn't explain what is returned (e.g., architecture name, reasoning). The tool is moderately complex (2 params, enums) but the description leaves substantial gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description adds general context about cost and performance but does not elaborate on how parameters influence the recommendation beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool recommends CPU architecture (ARM/x86) based on workload characteristics, considering cost and performance. The verb 'recommend' and resource 'CPU architecture' are specific, and the tool is distinct from all listed siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for workload-based architecture decisions but provides no explicit when-to-use, when-not-to-use, or alternative tools. Siblings are unrelated, so no confusion, but the lack of explicit guidance limits the score to 3.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_aws_referenceA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It adequately implies a read-only operation via 'get' and '조회', but does not disclose idempotency, rate limits, or whether the call has side effects. For a simple retrieval, this is sufficient but not exemplary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence clearly states purpose and key parameters. No extraneous information, front-loaded with the action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 2 simple enum parameters and no output schema, the description covers the essential inputs. Missing return value description, but given the tool's simplicity and sibling context, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters with descriptions ('서비스 유형', '서비스 규모'), and the description merely repeats those concepts without adding meaning. Since coverage is 100%, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb '조회' (retrieve) and resource 'AWS Well-Architected 기반 아키텍처 패턴', clearly stating the tool's purpose. It distinguishes from siblings like design_architecture, which implies custom design rather than retrieving existing patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. Does not mention when not to use it or provide context for selecting between this and sibling tools like get_scaling_strategy or recommend_cpu_arch.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.