Server Quality Checklist

Tool Scores

• get_statsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description gives high-level overview but lacks detail on output structure, performance, or data freshness. Output schema exists but not described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded, no wasted words. Efficiently communicates purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a parameterless tool with output schema, but could be improved by detailing output structure or providing example. Vague on 'crosswalk mappings'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters (0 params = baseline 4). Description adds meaning by specifying the content of statistics beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it returns database statistics: code counts per system and crosswalk mappings. This distinguishes it from siblings like browse_category, lookup_code, search_codes, translate_code which deal with individual codes or searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. Does not mention when to prefer get_stats over search_codes or browse_category.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that search is full-text, system defaults to all, and limit defaults to 20 with max 50. However, it does not mention case sensitivity, matching behavior, ordering, or whether the search is partial/fuzzy. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: one sentence for purpose, followed by a clean parameter list. No redundancy, efficiently front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description does not need to detail return structure. It covers required input semantics. Missing some context like sorting, but overall sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds thorough semantic meaning: explains 'query' with example, lists allowed 'system' values with note that empty means all, and clarifies 'limit' default and max. This compensates fully for the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search codes by clinical description using full-text search.' It specifies the verb 'Search', the resource 'codes', and the method 'full-text search'. This distinguishes it from siblings like 'browse_category' (browse by category) and 'lookup_code' (lookup specific code).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lacks explicit guidance on when to use this tool vs alternatives. It does not state when to prefer this over 'browse_category', 'get_stats', 'lookup_code', or 'translate_code'. No 'when-to-use' or 'use-this-over' statements are present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_codeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description should disclose behavioral traits. It mentions the function returns a description but does not cover error handling, rate limits, or side effects. The tool is simple, but additional transparency would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using a clear docstring format that front-loads the purpose and efficiently explains each parameter. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, the presence of an output schema, and the clear parameter descriptions, the description is nearly complete. It lacks information on edge cases like code not found or invalid system, but overall covers the essential requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description adds meaning by explaining the code parameter with examples and dot optionality, and the system parameter with valid values. This compensates for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up a medical code and returns its description, with examples and parameter explanations that distinguish it from sibling tools like search_codes and translate_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives, but the sibling names imply that lookup_code is for retrieving a description for a single code, while search_codes is for searching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• translate_codeA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions using GEMs crosswalk but does not disclose behavioral traits such as read-only nature, error handling on invalid codes, rate limits, or required authentication. It also does not describe what the output contains beyond the existence of an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence for the main purpose followed by parameter documentation. Every sentence is informative, with no wasted words. The structure is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters, no nested objects) and the presence of an output schema, the description is fully adequate. It explains the translation method, parameter formats, and auto-detection behavior. There are no gaps in context for an AI agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema: it explains the format of 'code' with examples ('250.00' for ICD-9, 'E119' for ICD-10-CM), specifies the allowed values for 'source_system' ('icd9cm' or 'icd10cm'), and clarifies that an empty value triggers auto-detection. Schema coverage is 0%, so the description fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Translate between ICD-9-CM and ICD-10-CM') and the method ('using GEMs crosswalk'). The verb 'Translate' and specific resource 'ICD-9-CM and ICD-10-CM' distinguish it from sibling tools that browse, get stats, lookup, or search codes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use by specifying the purpose (code translation) and notes auto-detection for the source system. While it does not explicitly state when not to use or name alternatives, the context of sibling tools makes the usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• browse_categoryA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral traits. It discloses defaults (limit 50, system icd10cm) and maximum limit (100), which is helpful, but does not explicitly state read-only behavior or side effects. The tool name 'browse' implies read-only, but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single-sentence purpose followed by clearly formatted parameter bullet points. Every sentence is useful, and the most critical information (purpose, prefix, system, limit) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description need not explain return values. It covers the core purpose, all parameters with defaults and constraints, and differentiates from siblings implicitly. The description is complete for a straightforward browsing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no property descriptions in schema), so the description must compensate. It provides detailed semantics: prefix examples ('E11', 'I25'), system values ('icd10cm', 'icd9cm', 'icd10who'), limit default (50) and max (100). This adds significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Browse all codes under a category prefix.' The verb 'browse' and resource 'codes under a category prefix' are specific and distinct from sibling tools like lookup_code (single code) and search_codes (query), making it easy for an AI agent to select correctly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through parameter explanations but does not explicitly state when to use this tool over alternatives. No when-not or where-else guidance is provided, so the agent must infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.