Server Quality Checklist

Tool Scores

• gobernador_map_cross_frameworkA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool is safe and non-mutating. The description adds valuable behavioral details: partial search capability, expected return format (JSON with mappings), and error handling (empty list with suggestions). No contradictions exist between description and annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but well-organized with explicit sections (purpose, args, returns, examples, error handling). It is front-loaded with the core purpose and each section contributes useful information. Minor redundancy could be trimmed, but overall it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description fully explains the return JSON format with an example structure. The tool is simple (one parameter) and the description covers purpose, parameters, return, examples, and error handling. Annotations provide safety context. The description is sufficiently complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for the single required parameter 'concept', including examples and constraints. The description goes beyond the schema by explaining that searches are partial and accepts both name or ID, and provides additional usage examples, adding meaningful value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching for a compliance concept in a 51-concept ontology and showing how it translates across three frameworks (Chile's Law 21.719, ISO/IEC 42001:2023, and the EU AI Act). It uses a specific verb ('Busca') and resource, and the focus on cross-framework mapping distinguishes it from sibling tools like audit text or data flow checking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context with example questions ('¿cómo se llama en el EU AI Act lo que la Ley 21.719 exige en materia de X?') and concrete use cases. It does not explicitly state when not to use the tool or mention alternatives, but the purpose is clear enough to guide appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gobernador_check_data_flowA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds behavioral context: it's a 'vista filtrada' of another tool, and returns a structured JSON. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured into sections: purpose, args, returns, examples, and when-not-to-use. Every sentence adds value with no redundancy. Front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter, no output schema, and comprehensive annotations, the description covers return format and provides examples. No output schema makes the return description necessary, and it is provided adequately. Minor improvement could include more edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a detailed description of the 'description' parameter. The main description reiterates the parameter's purpose but adds examples that illustrate usage. However, since schema already defines the parameter clearly, the description adds marginal semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool evaluates a data flow description focusing on international transfers, legal basis, and DPIA. The title 'Verificar flujo de datos' and description distinguish it from siblings like 'gobernador_audit_text' (general audit) and 'gobernador_generate_compliance_report' (AI system compliance).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use context ('útiles queries rápidas de "¿puedo mover estos datos aquí?"') and when-not-to-use ('No usar cuando: se necesita clasificar riesgo de un sistema de IA completo'). Names the alternative tool 'gobernador_generate_compliance_report'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gobernador_generate_compliance_reportA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explains that the tool internally runs the same engine as audit_text but adds an executive summary and deduplicates findings. It aligns with annotations (readOnlyHint, idempotentHint, destructiveHint) and provides additional context about aggregation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose, internal workings, argument details, examples, and a usage guideline. It is slightly verbose in repeating schema descriptions but overall each section adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all necessary aspects: purpose, differentiation from siblings, parameter details with examples, return structure (since no output schema), and usage guidelines. It provides sufficient information for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions, and the description adds further context: system_description requires at least 20 characters and should include purpose, data types, decisions, users, etc. Frameworks are clarified with allowed values and default behavior (all three if omitted).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a structured compliance report for AI systems, aggregating findings by severity and framework. It distinguishes itself from siblings by being the highest-level tool that adds executive summary and deduplicates findings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when not to use this tool (e.g., for checking a specific text fragment) and recommends gobernador_audit_text as a faster alternative. It also implies use cases like client reports or internal pre-release reviews.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gobernador_audit_textA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as read-only and idempotent. The description adds behavioral context by explaining it is a heuristic engine based on text patterns, does not replace legal advice, and points to relevant articles/clauses. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: purpose and disclaimer first, then parameters, return format (with JSON structure), usage examples, and finally when-not-to-use. Every sentence adds value, no fluff. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters and no output schema, the description provides a complete picture: explains what the tool does, how to use it (with examples), what the output looks like (JSON schema), and when to use an alternative. All relevant context is covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant value: character limits (10-20,000), language requirement (español), explanation of the optional framework parameter with enum values, and detailed examples of usage. This goes well beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('analiza') and resource ('descripción de un sistema de IA o flujo de datos'), specifies the compliance frameworks (Ley 21.719, ISO/IEC 42001:2023, EU AI Act), and distinguishes from sibling tools by noting that for executive reports one should use gobernador_generate_compliance_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when not to use ('No usar cuando: se necesita un informe ejecutivo completo'), provides a specific alternative tool ('usar gobernador_generate_compliance_report'), and includes a disclaimer that it does not replace legal advice. This provides clear guidance on appropriate usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.