Server Quality Checklist

Tool Scores

• validate_messageA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that it returns structured issues, which indicates a non-throwing behavior and provides expectations. It also clarifies that FHIR validation uses a minimal US Core profile. However, with no annotations, it doesn't disclose error handling, permissions, or side effects. This is moderate but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the tool's purpose and output. No filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool validates two different data formats and has an optional 'profile' parameter, but the description does not explain the profile's purpose or how validation results are structured. It also lacks guidance on input format for payload. Given the absence of output schema and annotations, the description is insufficient for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not explain the 'payload' or 'profile' parameters at all. The only parameter hinted at is 'kind' by mentioning HL7 v2 and FHIR, but the enum already covers that. The description adds no semantic value for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Valida') and clearly states the resources (HL7 v2 with segments/required fields, FHIR Bundle with minimum US Core profile) and the output (structured issues). This distinguishes it from siblings like map_v2_to_fhir, explain_error, and parse_hl7v2.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: this tool is for validation, not mapping, parsing, or error explanation. However, it does not explicitly mention alternatives or exclusion criteria, so it doesn't achieve a 5. The user can infer when to use it based on the verb and resource types.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_errorA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explains the output components (readable location, HL7 table meanings, actionable hint) which gives some transparency, but it does not explicitly state whether the tool is read-only or has any side effects, permissions, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the tool's purpose and key deliverables without excess words. Every phrase earns its place, and the structure is efficient for an agent to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives the essential context: it consumes a validation issue and produces a human explanation with specific elements. However, without an output schema or annotations, it does not specify the return format (e.g., string, object) or any potential error conditions, leaving some gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one 'issue' object with 0% schema description coverage, so the description must compensate. The description mentions 'issue de validación' and the output components (location, table meaning, hint), but it does not explain the individual properties (severity, code, location, message) or how they relate to the transformation, leaving the parameter semantics underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Convierte' (converts) and identifies the resource as 'un issue de validación' (a validation issue), clearly stating the tool's function: turning validation issues into human-readable explanations. It distinguishes itself from sibling tools (parse, validate, map) by focusing on explaining errors rather than processing messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool through the phrase 'Convierte un issue de validación', suggesting it is applied after validation has produced an issue. However, it does not explicitly state when not to use it or mention alternative tools, leaving usage boundaries implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• map_v2_to_fhirA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It goes beyond simple mapping by explaining that it validates against US Core and explains each issue, which is useful context. However, it does not discuss error handling, limitations, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the core action and no filler. Every phrase adds meaningful information about mapping, validation, and issue explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential mapping and validation behavior and provides a high-level output description, but it omits details on mapId and the fhirVersion options. Given no output schema or annotations, it is reasonably complete but has notable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It identifies 'message' as an HL7 v2 message and mentions a declarative map, but it fails to explain mapId or fhirVersion. Moreover, it states 'FHIR R4' while the schema allows R6, potentially misleading the agent about the fhirVersion parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool maps an HL7 v2 message to an FHIR R4 Bundle using a declarative map and validates the result. It distinguishes this from siblings like validate_message or parse_hl7v2 by specifying the conversion and validation action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for mapping HL7 v2 to FHIR and validating, but it does not explicitly state when to use this tool over siblings like validate_message or explain_error. No exclusions or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_hl7v2A
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key behavioral detail—separators are read from MSH-1/MSH-2—which adds technical context. However, it does not mention error behavior, strictness, or other potential side effects, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary action ('Parsea un mensaje HL7 v2 a un AST tipado') and then provides a necessary technical detail. No wasted words; every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simple signature (one parameter, no output schema), the description is reasonably complete. It describes both the input and the output format (typed AST) and a critical parsing behavior (separators from MSH-1/MSH-2). It could mention error handling or validation behavior, but the core functionality is sufficiently conveyed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only one parameter ('message') with zero coverage in the description, so the description must compensate. It does clarify that the 'message' is an HL7 v2 message to be parsed and adds context about separator handling, which gives the parameter meaningful semantics despite lacking constraints or format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Parsea' = parses) and resource ('mensaje HL7 v2'), and details the output as a typed AST with segments, fields, and components. This distinguishes it from siblings like map_v2_to_fhir (mapping) and validate_message (validation), making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by explaining what the tool does, but it does not explicitly state when to use it or provide exclusion criteria or alternatives. Sibling tool names suggest context, but no direct guidance is given, so usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.