Server Quality Checklist

Tool Scores

• list_module_versionsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description is the sole source of behavioral information. It discloses the output format (JSON object) and sorting order (semantic version descending), but omits details such as authentication requirements or behavior when no tags exist. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args/Returns sections and is relatively concise. However, the opening sentence and the 'Devuelve todos los tags' line are somewhat redundant, costing a minor efficiency point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-parameter list tool, the description provides essential information: purpose, output format, and sorting. However, it lacks usage guidance and discussion of edge cases, making it not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions, and the description adds only a brief explanation with examples ('service_name: Nombre del servicio...'). This gives some context beyond the schema title but does not fully define the valid values or how the service name maps to a module.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists available versions (git tags) of a Terraform module, using a specific verb and resource. It also mentions sorting by semantic version, which distinguishes it from sibling tools like get_module or search_modules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools. It doesn't mention any prerequisites, alternatives, or exclusions, so an agent would have to infer usage from the function name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scaffold_terraformA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses important behavioral traits: the provider always assumes IAM_ROLE_NAME, static credentials are not an option, and aws_account_id is filled automatically. It also documents the return format. However, it does not mention whether existing files are overwritten or if a module must already exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a compact file list, IAM role constraint, and Args/Returns sections. Every sentence adds necessary context, and there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with an output schema, the description covers file structure, IAM role behavior, and return format. It lacks edge cases like overwrite behavior or module existence prerequisites, but is otherwise adequate for a scaffolding task.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining service_name with an example and variables as an optional dict for preloading terraform.tfvars. Both parameters are covered, though the exact structure of variables could be more detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Genera' and identifies the resource as a complete Terraform configuration using a module. It enumerates the standard files created, making the tool's scope clear and distinguishing it from sibling read/search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the tool's function clear but provides no explicit guidance on when to use this tool versus alternatives like search_modules or get_module. Usage is implied rather than explicitly contrasted with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_moduleA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It clearly discloses the returned data (readme, variables, outputs, last version) and the repo-name construction rule. It doesn't cover error cases or authentication, but it adds significant behavioral context beyond a bare 'get'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The text is well-structured: a lead sentence, a short bullet list of retrieved items, an Args block, and a Returns block. It is concise with no redundant content; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read operation with an output schema, the description explains purpose, parameter format, and return summary. It is complete enough for correct selection and invocation, especially given the output schema already specifies return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only service_name with 0% description coverage. The description compensates fully by providing examples ('vpc', 'ec2') and explaining the naming pattern 'terraform-aws-module-{service_name}', which is exactly the semantic needed for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Obtiene información detallada de un módulo de Terraform concreto.' It clearly lists the retrieved artifacts (README, variables.tf, outputs.tf, version tag) and is distinct from sibling tools like search_modules or list_module_versions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for fetching detailed module info, but it never explicitly contrasts with alternatives like list_module_versions or search_modules. Guidance on when to use this tool vs. siblings is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_backend_configA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and succeeds by disclosing the fallback behavior, the priority of bucket/region over the environment variable, and the rule to never generate commented-out or placeholder backends. It doesn't cover potential errors but provides essential behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a purpose sentence, a behavior paragraph, and an Args list. It is moderately detailed but each sentence adds value, and the formatting aids scanning. Slightly longer than necessary but not padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and an interactive workflow, the description covers purpose, all parameters, the return value (JSON with backend block and destination file), and the fallback behavior. It is complete and leaves no major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the Args section is critical. It explains each parameter with examples (team: 'platform', project: 'onboarding') and explicitly states that bucket/region take priority over BACKEND_CONFIG, fully compensating for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Obtiene la configuración del backend S3 de un proyecto Terraform', a specific verb and resource. It clearly differentiates from sibling tools like search_modules and scaffold_terraform by focusing on backend S3 configuration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear workflow guidance: if BACKEND_CONFIG is not defined, the tool returns `needs_user_input`, prompting the agent to ask the user for bucket/region and re-call with those values. It does not explicitly name alternatives, but the context is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_modulesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral burden. It discloses the search prefix, optional query filtering, and the exact return fields. It doesn't mention authentication, rate limits, or pagination, but it's a read-only search operation and the behavior is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-sentence purpose, a clarification of filtering, and explicit Args/Returns sections. Every sentence serves a purpose, and it's appropriately brief for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one optional parameter, an output schema, and clear context. The description explains the prefix, query behavior, and return fields, making it complete for an agent to understand what the tool does and what it returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for the 'query' parameter and coverage is 0%. The description explicitly explains the parameter as an optional filter to narrow results by service name, fully compensating for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool searches for Terraform modules in a GitHub organization, listing repositories with the prefix 'terraform-aws-module-'. This specific verb+resource combination (search/list modules) distinguishes it from siblings like get_module or list_module_versions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the default behavior (list all matching prefix) and how an optional query filters by service name. It implies its role as the discovery tool among siblings, though it does not explicitly mention when not to use it or direct users to an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.