Server Quality Checklist

Tool Scores

• generar_checklist_auditoriaC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says the tool generates a reproducible evidence list, but it does not explain whether the operation has side effects, how the boolean parameters affect behavior, what output format to expect, or whether anything is persisted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler words. It is front-loaded with the core purpose, though it lacks any structured elaboration about parameters or output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given six parameters, no output schema, and no annotations, this description is far too sparse to allow an agent to invoke the tool correctly. It doesn't explain how the inputs map to an audit checklist or what the returned 'lista de evidencias' contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention any of the six input parameters. It does not clarify the role of 'name', the meaning of 'language', or how 'sellsOnline', 'usesProviders', 'storesSensitiveData', and 'processesPersonalData' shape the generated checklist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action and resource: 'Genera una lista reproducible de evidencias para auditoría.' This is specific enough to distinguish it from siblings that search regulations, consult obligations, or audit repositories, though it does not explicitly name a sibling or scope the audit type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus the sibling tools such as 'auditar_repositorio' or 'evaluar_proyecto.' The description implies use when an audit evidence list is needed, but it provides no exclusions, prerequisites, or context for choosing among alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verificar_vigenciaC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses the return payload (status and verification date) but says nothing about mutation guarantees, error behavior, permissions, or what happens when an id is not found. For a verification-style tool this is only partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler; every word contributes to the core message. It is appropriately minimal for a simple tool, even though it could convey more useful content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without annotations or an output schema, the description should provide enough information for correct invocation. It omits id semantics, language behavior, and failure handling, and never explicitly connects the tool to the apparent legal-norm domain suggested by the sibling names.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain what 'id' identifies or how 'language' affects the result. Neither 'fuente' nor the return statement clarifies parameter meaning or expected formats. The agent is left with only the property names and schema constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific return verb ('Devuelve') and identifies the resource ('estado y fecha de verificación de una fuente'), making the core function clear. It is not a tautology of the name and its focus on status/verification date distinguishes it from siblings like buscar_normativa or auditar_repositorio, though 'fuente' is somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as consultar_obligacion or buscar_normativa. There are no stated conditions, exclusions, or prerequisites. An agent would need to infer usage from the tool name and sibling context, which the description itself does not support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluar_proyectoC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It indicates the tool generates preliminary risks and controls, but does not clarify whether it persists results, requires special permissions, or produces AI-generated output that should be reviewed. The word 'preliminares' adds a small degree of context but not enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler. It is appropriately front-loaded with the main action and result, though it sacrifices helpful detail for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, no annotations, and six parameters with zero schema documentation, this description is incomplete. An agent does not know the output format, whether the operation has side effects, how parameters affect results, or what 'preliminary' means operationally.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate by explaining parameter meaning, but it does not. The booleans are reasonably self-explanatory from their names, but the description fails to confirm how they influence the generated risks/controls or what 'name' and 'language' fully contribute.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Genera') and resource ('riesgos y controles preliminares para un proyecto'), which distinguishes it from siblings focused on regulation lookup and auditing. It could be slightly more explicit about the supporting inputs, but the core purpose is understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to prefer this tool over siblings like generar_checklist_auditoria or auditar_repositorio. The context implies it is for project risk evaluation, but there are no stated conditions, exclusions, or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consultar_obligacionB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of disclosing behavior, yet it only states the core operation. It does not mention return format, whether the operation is read-only, error behavior, or how the language parameter affects results. The Spanish verb 'Consulta' weakly implies a read operation, but critical behavioral context is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with zero filler, stating the verb and resource immediately. It is appropriately concise for a simple lookup tool, though the brevity leaves out useful guidance that would push it higher.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description should explain what the call returns and any caveats; it does neither. The language parameter's effect on the response is also absent. The core purpose is clear, but an agent lacks essential information about response shape and optional behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only clarifies that 'id' is the identifier. The 'language' parameter and its es/en enum are completely unexplained. This is insufficient compensation for the complete lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Consulta'), a precise resource ('ficha normativa'), and a clear lookup method ('por identificador'). This differentiates it from siblings like buscar_normativa (text search) and verificar_vigencia (validity check), so an agent can tell them apart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'por identificador' implies this tool should be used when the agent already has a regulatory record ID, but it never explicitly states when to prefer it over alternatives or mentions exclusions. Usage guidance is present but only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• auditar_repositorioA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does disclose key behavioral traits: read-only, local, static, and bounded by safe limits. This is valuable safety information, though it does not detail what those limits are or how failures are handled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence communicates the core purpose and behavioral character with no filler. Every phrase earns its place, even if 'límites seguros' is somewhat vague.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no output schema, and no annotations, the description is too sparse for an agent to call the tool reliably without guessing. It does not mention return format, how to use path, what dependencyScan implies, or the meaning of the safety limits in practice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate. Phrases like 'límites seguros' hint at timeout and max* parameters but explain none of the 7 parameters, including format, dependencyScan, or path semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('auditoría estática local y de solo lectura') on a clear resource ('repositorio'), using a distinct verb and scope. It differentiates from sibling tools like evaluar_proyecto or generar_checklist_auditoria by focusing on a local, read-only repository audit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: an agent would select this tool when needing to audit a repository. However, there is no explicit guidance on when to prefer this over siblings, no exclusion criteria, and no mention of prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buscar_normativaA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It indicates a search/read operation and mentions the language enum, but it does not describe result format, scope limitations, pagination, or prerequisites. 'Verificable' hints at source quality but is ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: the primary action is front-loaded, and the second sentence provides immediately actionable parameter guidance. Every word earns its place without filer.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple three-parameter search tool, the core calling information is present, including language selection. However, there is no output schema and the description does not state what the tool returns, nor clarify how an empty call behaves given that all parameters are optional. This leaves some ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are absent (0% schema description coverage), but the description maps 'texto' to the query parameter and 'tema' to the topic parameter. The phrase 'Use language=es o language=en' explicitly explains the language enum. It does not clarify how query and topic interact or whether at least one is needed, but it compensates for the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb, 'Busca', and a clear resource, 'normativa ecuatoriana', with two search dimensions: text and topic. This clearly distinguishes it from sibling tools like generar_checklist_auditoria or auditar_repositorio, which are not search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is implied: use this when searching Ecuadorian regulations by text or topic. However, there is no explicit comparison to siblings such as consultar_obligacion or verificar_vigencia, and the language instruction is parameter guidance rather than when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.