Server Quality Checklist

Tool Scores

• code_diagnosisA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations declare readOnlyHint=true and destructiveHint=false, the description adds valuable behavioral context: it discloses NLP capabilities (negation detection, historical mentions, family history, severity assessment) and output format (ranked candidates with confidence scores) that annotations do not cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two efficient sentences. The first sentence front-loads the core function (extraction and coding), while the second adds capability details (negations, severity) and output structure. Zero redundant words or tautologies.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a clinical NLP tool of moderate complexity, the description adequately covers the input domain, processing capabilities, and return format (compensating for the missing output schema). It could improve by mentioning input length constraints or privacy considerations given the PHI nature of clinical text, though the presence of the 'anonymize' sibling mitigates this gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the parameter details are fully documented in the schema itself. The description references 'clinical text' and coding systems which map to the text and output_system parameters, but does not add substantial semantic meaning beyond what the structured schema already provides, meeting the baseline for high-coverage schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Extract medical diagnoses from clinical text and map them to ICD-10-CM or ICD-11 codes,' providing specific verbs (extract, map) and resources (diagnoses, clinical text). It implicitly distinguishes from siblings like get_code/search_codes by emphasizing unstructured text processing vs. direct code lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'clinical text' (progress notes, discharge summaries), signaling when to use this over lookup-based siblings. However, it lacks explicit guidance contrasting with alternatives like 'use this when you have free text rather than a known diagnosis term.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_icd11_codeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare read-only/idempotent/safe properties, so the description appropriately focuses on disclosing return data structure (descriptions, URIs, hierarchies, crosswalks) rather than safety; however, it omits behavioral details like caching, rate limits, or 'not found' error behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence that front-loads the action ('Get comprehensive details') and efficiently lists return components without redundancy; every clause serves to clarify the scope of 'comprehensive details.'

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description compensates effectively by enumerating the specific data fields returned (hierarchy, crosswalks, etc.); for a single-parameter lookup tool with strong annotations, this provides sufficient context despite omitting error-handling specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage and the schema providing clear examples ('5A11', 'BA00'), the description meets the baseline expectation by referencing 'ICD-11 code' but adds minimal semantic clarification beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('ICD-11 code'), clearly distinguishing it from siblings like 'get_code' (generic) and 'search_icd11_codes' (search functionality) by emphasizing 'specific' code retrieval and enumerating exact return fields (Foundation URI, synonyms, hierarchy, crosswalks).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The term 'specific ICD-11 code' implies usage when an exact code is known, contrasting with the sibling 'search_icd11_codes', but the description lacks explicit when-to-use/when-not-to-use guidance or explicit naming of alternatives for code discovery scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only, idempotent, non-destructive behavior. The description adds valuable behavioral context by disclosing return values ('matching codes with descriptions and billable status') which compensates for the missing output schema, and specifies the 2025 version of the directory being searched.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences are perfectly weighted: first establishes core action, second discloses return structure (critical given no output schema), third provides usage context. No redundancy or wasted words. Information is front-loaded with the essential operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple search interface (3 params, 100% schema coverage) and strong annotations, the description is appropriately complete. It compensates for the missing output schema by describing return values, covers the specific version (ICD-10-CM 2025), and provides adequate context for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description mentions 'by description text' which aligns with the query parameter but does not add additional semantic context, examples, or usage notes beyond what the schema already provides for the three parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Search' with clear resource 'ICD-10-CM 2025 code directory' and scope 'by description text'. It distinguishes from siblings like search_icd11_codes and get_code by specifying the ICD-10-CM 2025 version and text-based search approach.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it is 'Useful for finding specific diagnosis codes by keyword', providing implied usage context. However, it lacks explicit when-to-use guidance or named alternatives (e.g., does not mention to use get_code when the exact code is known).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• anonymizeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only and non-destructive behavior; the description adds valuable specifics about what entity types are detected (MRNs, ages, phone numbers) and the exact replacement format ('[NAME]', '[DATE]'). The HIPAA compliance claim adds important regulatory context not found in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two efficiently structured sentences with zero redundancy. The first states the core operation; the second details the specific PHI categories, replacement patterns, and compliance standard. Every clause conveys distinct, necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter text processing tool without an output schema, the description adequately covers the transformation logic (what gets replaced and with what). It would be marginally improved by explicitly stating that the tool returns the anonymized string, though this is reasonably implied by the 'replaces... with' phrasing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input parameter 'text' is fully documented in the schema itself. The description implies the content should be clinical text but does not add additional semantic constraints, format requirements, or examples beyond what the schema provides, warranting the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Detect and mask') and clearly identifies the resource ('Protected Health Information in clinical text'). It effectively distinguishes itself from sibling code-lookup tools (get_code, search_codes, etc.) by specifying the domain is clinical text de-identification rather than medical coding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('HIPAA-compliant de-identification') and lists specific entity types handled (names, dates, SSNs, etc.). While it doesn't explicitly name alternatives to avoid, the domain focus makes the appropriate use case unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_icd11_codesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, covering safety behavior. The description adds valuable functional context by specifying the return format: 'Returns matching codes with descriptions and Foundation URIs,' compensating for the lack of an output schema. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of three efficient sentences, each earning its place: (1) action and method, (2) return value disclosure, (3) usage context. It is front-loaded with the core action and contains no redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of annotations (covering safety/idempotency), 100% schema coverage, and the explicit mention of return values (codes, descriptions, Foundation URIs) in the description, the tool is well-documented. Minor gap: pagination behavior or fuzzy vs exact matching semantics are not specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input parameters are fully documented in the schema itself (query description covers the search text, limit covers the range). The description reinforces this with phrases like 'by description text' and 'by keyword,' but does not add syntax details or examples beyond the schema definitions, warranting the baseline score for high-coverage schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool 'Search[es] the ICD-11 code directory by description text' - a specific verb + resource + method combination. It distinguishes from sibling tools like get_icd11_code by specifying this is a search-by-text operation rather than a direct lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context stating it is 'Useful for finding specific ICD-11 diagnosis codes by keyword,' indicating when to use the tool. However, it does not explicitly name alternative tools (e.g., 'use get_icd11_code if you already know the exact code') or state exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_codeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, non-destructive safety. The description adds valuable behavioral context by detailing exactly what comprehensive data is returned (billable flags, medical ontologies like SNOMED/UMLS, hierarchical relationships), helping the agent understand the tool's informational scope beyond generic 'details'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, dense sentence efficiently packs six distinct data categories (descriptions, billable status, two synonym types, hierarchy, classification) without filler. Every clause serves to specify the return payload or resource type.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description excellently compensates by enumerating the specific data fields and relationships returned (parent/child hierarchy, chapter classification, external ontologies). For a simple single-parameter lookup tool, this provides sufficient context for the agent to understand both input requirements and output value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is appropriately met. The description reinforces that the parameter should be a 'specific' code but does not add syntactic guidance, validation rules, or semantic nuances beyond what the schema's examples ('E11.9', 'I10') already provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Get' with clear resource 'ICD-10-CM code' and enumerates specific data returned (billable status, SNOMED CT/UMLS synonyms, hierarchy, chapter). It distinguishes from sibling 'get_icd11_code' by explicitly specifying 'ICD-10-CM' and from 'search_codes' by emphasizing 'specific' code retrieval versus search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While it lacks explicit 'when to use' statements, the phrase 'specific ICD-10-CM code' combined with the single required 'code' parameter clearly signals this is for lookup when the code is already known, implicitly contrasting with the 'search_codes' sibling. The ICD-10-CM specificity clearly differentiates from the ICD-11 variant.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.