Server Quality Checklist

Tool Scores

• query_api_knowledgeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No hay anotaciones, por lo que la descripción debe cargar con la transparencia de comportamiento. Dice que sintetiza conocimiento, lo cual aporta algo, pero no aclara si la búsqueda es de solo lectura, qué tipo de respuesta devuelve, si el conocimiento puede estar incompleto, ni cómo se relaciona con las especificaciones ya aprendidas.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  La descripción es una única oración clara y sin contenido redundante. Comunica el propósito y el contexto principal de forma compacta, aunque sacrifica espacio para matices de uso y comportamiento.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Para una herramienta de consulta con dos parámetros y esquema completo, la descripción alcanza lo esencial: qué hace y con qué objetivo. Pero al no existir esquema de salida ni anotaciones, y al competir con varias herramientas documentales cercanas, faltan detalles sobre resultado, alcance y límites de la 'síntesis' prometida.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  La cobertura de esquema es del 100%, por lo que los parámetros query y specId ya están documentados en el esquema. La descripción no añade detalles nuevos sobre el formato de la consulta ni sobre cómo afecta specId al resultado, por lo que se mantiene la línea base.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  La descripción nombra una acción específica ('consulta y sintetiza') y un recurso concreto: el conocimiento transversal de APIs OpenAPI aprendidas. Se diferencia razonablemente de herramientas como list_specs o get_endpoint_doc, aunque no delimita explícitamente su frontera con ellas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  La descripción indica un contexto de uso claro: resolver dudas de negocio o integración y trabajar sobre conocimiento transversal de varias APIs. Sin embargo, no señala cuándo preferir esta herramienta sobre alternativas como search_docs o get_schema_doc, ni menciona exclusiones o cuándo no usarla.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_integration_codeB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the full behavioral burden, and it only claims 'producción' (production-ready) quality without explaining output format, side effects, errors, prerequisites, or determinism. With no output schema, the absence of any statement about what the tool actually returns is a real gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, efficient sentence that front-loads the action verb 'Genera', names the languages, and scopes the tool to a specific endpoint. Zero filler, every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple five-parameter tool with 100% schema coverage this is adequate at a minimum-viable level, but the absence of an output schema and annotations puts more weight on the description than it handles: it doesn't state the return value shape, when to use the tool among its siblings, or whether runtime checks depend on a valid spec/endpoint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3 even though the description adds no parameter-level detail. The languages listed in the prose merely echo the enum values of the 'language' parameter, and 'segmento específico' matches 'path', so the description adds almost nothing beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Genera') and a clear resource ('un snippet/cliente de código listo para producción... para un endpoint específico'), naming the target languages. Since all sibling tools only list, search, read, validate, or query knowledge, this is unmistakably the only code-generation tool, so it distinguishes itself implicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to choose this tool versus any sibling, nor does it mention exclusions or prerequisites (e.g., needing a spec or a valid endpoint first). The usage context is only implied by the purpose sentence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpoint_docB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool performs a read-only retrieval and specifically highlights that responses are 'dereferenciadas' (resolving $refs), which adds useful behavioral detail. However, it does not mention rate limits, authentication, or what a complete response structure actually looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise, dense sentence that starts with the verb and quickly enumerates the returned artifacts. No fluff, redundant phrases, or repetitive wording. It is front-loaded with the core action and object.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with just 3 well-documented parameters and no output schema, the description sufficiently states what the tool retrieves and the key behavioral trait of de-referencing. The lack of an explicit when-to-use contrast is a gap, but that falls largely under usage guidance rather than overall completeness for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all three parameters with detailed descriptions and an enum for method, achieving 100% schema description coverage. The description adds no extra information beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the specific verb 'Obtiene' and the exact resource: complete documentation including parameters, request body, and de-referenced responses for a specific endpoint. It clearly differentiates from sibling tools like list_specs or get_schema_doc through the 'endpoint específico' phrase, though it does not explicitly name the alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool instead of alternatives such as search_docs or get_schema_doc. It merely describes what it retrieves, leaving the selection criteria entirely to the agent to infer. There are no exclusions, prerequisites, or explicit usage contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It conveys that this is a search-like operation across the full learned corpus, which is useful, but it does not mention whether the operation is strictly read-only, how results are ordered or shaped, or whether pagination or auth constraints apply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One front-loaded sentence with no filler. It states the action, the resources being searched, the corpus scope, and the keyword mechanism in a compact and immediately parseable way.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The input side is well covered by the schema, and the description gives the high-level behavior. But there is no output schema and no explanation of what results look like, and the boundary with sibling tools, especially query_api_knowledge, is not clarified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters like query, tag, limit, and specId already have clear descriptions and examples. The description only reinforces the keyword-search mechanism without adding meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the verb ('Busca'), the resource types ('endpoints, operaciones, etiquetas y modelos de datos'), and the scope ('todas las especificaciones OpenAPI aprendidas'). It distinguishes itself from list_specs and the get_doc siblings, though it does not explicitly address possible overlap with query_api_knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: use this tool for keyword-based search across learned OpenAPI specifications. However, there are no explicit when-to-use or when-not-to-use conditions, and no sibling alternatives are named, so an agent has to infer differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schema_docA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It discloses that the schema is dereferenced and enumerates the returned content (properties, types, required fields, enums), which is useful. It does not mention error cases, optional specId handling, or any resolution side effects, leaving moderate ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence that front-loads the main purpose and appends the relevant definition details. No filler or redundant content. It is compact without losing the essential scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool, the description gives the core information: what is fetched and what the return content includes. There is no output schema, so a bit more detail on the response format or specId behavior would help, but the description is substantially sufficient for an agent to decide to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents both parameters fully (coverage 100%), so baseline 3 applies. The description adds little about parameter semantics beyond implying schemaName is the thing to look up; no extra detail about how specId interacts is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (schema/data model definition) and what the tool returns (properties, types, required fields, enums). It does not explicitly differentiate from siblings like get_endpoint_doc or get_security_schemes, but the resource type is distinct enough that an agent can infer its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied: use this tool when you need a schema definition (e.g., 'LoginRequestDto' or 'MemberBalanceResponseDto'). There is no explicit guidance about when not to use it or which sibling alternative to prefer, but the purpose itself gives reasonable inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_security_schemesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It does indicate a read-only 'obtiene' operation and states the source of the schemes (OpenAPI specifications). It does not disclose behaviors such as error handling, pagination, or what happens when specId is omitted — although the schema notes that behavior, so it is not completely hidden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler. It immediately states the action and includes a parenthetical list of scheme types that adds useful detail without bloat. Structurally front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple read-only tool with one optional parameter and no output schema, so full context is largely achieved. The description covers what is returned, and the schema covers the parameter behavior. The only missing piece is an explicit statement that an omission of specId returns schemes across all specs, but that is already in the schema description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, specId, has 100% schema description coverage ('opcional, si se omite lista todos'). The main description does not add any additional parameter detail beyond the schema baseline. The mention of scheme types (Bearer token, API Key, OAuth2) describes output concepts, not parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'obtiene' (gets) and 'esquemas de autenticación y seguridad' (authentication/security schemes), and names concrete types (Bearer token, API Key, OAuth2). It is clearly distinct from siblings like get_endpoint_doc or get_schema_doc, but does not explicitly distinguish itself by name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: this tool is for retrieving security schemes from OpenAPI specs. However, it never explicitly explains when to use this tool versus a sibling, such as get_endpoint_doc or get_schema_doc, which may also provide security-related information. Guidance is functional but not direct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_payloadA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It says validation occurs, but does not state whether it is read-only, what happens on invalid or valid payloads, whether errors are returned as exceptions, arrays, or booleans, or if the call has any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It starts with the key verb, names the involved resource and criteria, and closes with the temporal context. Very easy to read.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so return values and possible error behavior must be described. The current description only states that validation happens; it leaves unclear whether the tool returns a boolean, an error list, or something else.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the parameter descriptions in the schema provide the needed meaning: schemaName, payload, and specId. The description adds no per-parameter semantics beyond what is already in the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses clear, specific verb ('Valida') and resource ('payload JSON...contra un schema OpenAPI') and even enumerates what is validated: structure, required fields, data types. This clearly distinguishes validate_payload from sibling tools that list, search, get, or generate code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear temporal context: 'antes de generar código', so an agent knows to call it before generate_integration_code. It does not mention explicit exclusions or alternatives, but the timing signal is quite useful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_specsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided, so the description carries the full burden. It does disclose that the specs are 'aprendidas y dereferenciadas en memoria' and that servers and versions are included, which is useful. It does not explain any potential limitations, whether it requires existing in-memory data, or what happens if no specs exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Exactly one sentence with no filler. It includes the resource type, file extensions, the in-memory state, and the return components in a compact, front-loaded way.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless listing tool, the description is sufficiently complete: it names the entities, their formats, their state, and the included properties. Minor gaps such as the exact response envelope or error behavior exist, but they are not critical for this simple operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description has very little burden here. The schema already covers all parameters. The description's mention of servers and versions gives a hint about the result shape, but no parameter semantics are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description immediately states the verb ('Lista'), the resource ('todas las especificaciones OpenAPI/Swagger'), and the scope ('aprendidas y dereferenciadas en memoria'). It clearly differentiates this from sibling tools like get_schema_doc and get_endpoint_doc by being a list operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says what the tool does but gives no guidance on when to use it versus the sibling tools, such as get_schema_doc or get_endpoint_doc. An agent must infer that this is the right choice for enumerating available specs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.