Server Quality Checklist

Tool Scores

• get_historyA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds implementation details (endpoint patterns) beyond the readOnlyHint annotation, but does not explain what the history response contains (e.g., version entries, ordering) or pagination behavior beyond the count parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with the primary purpose, no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description is mostly complete for a read operation with readOnlyHint, but lacks details about the return format (no output schema) and does not mention that the response is typically a Bundle of history entries, which would aid an agent in processing the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptive parameter descriptions, but the tool description adds meaningful context by explaining the three usage modes and how parameters relate (e.g., id requires resourceType), which goes beyond the schema's individual field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches version history and specifies three distinct modes based on parameter combinations (with id+resourceType, only resourceType, neither). This distinguishes it from siblings like search_fhir and read_fhir, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides explicit context for when to use each combination of parameters, but does not indicate when NOT to use the tool or mention alternative tools for similar tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• patient_everythingA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true. The description adds the behavioral context that it fetches a large bundle of resources, but does not detail pagination or performance implications. Since annotations cover the safety profile, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence defines purpose and endpoint; the second gives actionable parameter guidance. It is front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that returns a large bundle of resources, the description covers the key points: what is returned, optional filters, and the endpoint. Without an output schema, it could mention pagination or response format, but it is sufficient for a read-only operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so baseline is 3. The description adds value by explaining that `types` and `since` are recommended for limiting response size, providing context beyond the schema descriptions. This justifies a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch all resources in a patient's compartment' with specific examples (conditions, observations, etc.) and the FHIR endpoint. It distinguishes from sibling tools like read_fhir (single resource) and search_fhir (query-based).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description recommends using `types` and `since` to keep responses manageable, providing explicit guidance for efficient use. It implies this is for comprehensive patient data retrieval but does not explicitly contrast with siblings for when to use alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fhirA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true. The description adds the HTTP method and URL pattern (GET /{resourceType}/{id}), providing additional behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (read by type and ID), the description is complete. It covers the purpose, HTTP method, and URL pattern. Output schema is not needed for such a tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters having descriptions. The description reiterates 'by type and id' but does not add new meaning beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read a single resource by type and id', with a specific verb and resource. It distinguishes from sibling tools like search_fhir and patient_everything which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to read a single resource) by specifying 'Read a single resource by type and id'. It does not explicitly list alternatives but is clear in context of siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capabilitiesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and description adds that it returns a compact summary of resource types, interactions, search parameters, and operations. No contradictions. Slightly enhanced beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with action ('Fetch'), no wasted words. Perfectly structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, annotations, and no output schema, the description sufficiently explains the return value (compact summary of capabilities). Complete enough for a simple read-only discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so no need for parameter explanation. Schema coverage is 100%. Baseline score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches the CapabilityStatement and returns a compact summary of capabilities. Specifies verb 'Fetch' and resource 'FHIR server's CapabilityStatement'. Differentiates from sibling tools like search_fhir, read_fhir etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this first to discover what the server supports', providing clear usage context. Lacks explicit when-not or alternative tools but is adequate for a discovery tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_fhirA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description confirms the operation is read-only (no save), aligning with readOnlyHint annotation. It adds behavioral context about the endpoint and required structure (resourceType), surpassing annotation detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose and endpoint. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks description of the return value (e.g., OperationOutcome). While sufficient given no output schema, adding expected response type would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, schema already defines the param. Description adds critical constraint that resource must include resourceType, which is not in schema description, enhancing clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates a FHIR resource against server profiles without saving, using a specific HTTP method and endpoint. It distinguishes from sibling tools like search or read by focusing on validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this tool before create_fhir or update_fhir to catch errors early, providing clear context for when to use it over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_fhirA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the tool is safe for reads. The description adds valuable behavioral context: HTTP GET method, parameter passthrough, pagination behavior, and response size control. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The first sentence states the core action and HTTP method, the second provides crucial usage tips. Information is front-loaded and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description explains the pagination response (hasNextPage flag). For a search tool with moderate complexity, it covers essential aspects: parameters, pagination, response size. Could mention more about return format but is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 3 parameters with descriptions. The description adds extra meaning: params can include FHIR chaining, _include, _revinclude, and mentions _elements, _summary, _count which are not in schema but are related. Adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches resources of a given type via GET, and the title 'Search FHIR resources' reinforces this. It distinguishes from siblings like read_fhir (single resource retrieval) and patient_everything (specific patient bundle).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use and how: pass parameters as-is, use _elements/_summary for small responses, paginate with _count and hasNextPage flag. It doesn't explicitly state when not to use but siblings imply alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.