Server Quality Checklist

Tool Scores

• lookup_bundlingC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose whether the tool is read-only, what it returns, or any side effects. For a check tool, this is minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence directly stating the tool's purpose with no extraneous information. Efficient for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema or description of return values. An agent cannot know what the tool returns (boolean, list of bundled pairs, etc.). Missing context on bundling rules.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter is fully described in the input schema (CPT codes array). The description adds no additional meaning beyond the schema, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks if procedure codes are bundled together, distinguishing it from sibling lookup tools (e.g., lookup_cpt for code details). However, it lacks specificity on what 'bundled' means.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives like lookup_cpt or lookup_payer. Missing information on prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_payerC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose behavioral traits such as read-only nature, authentication requirements, or side effects. The verb 'look up' implies read-only but is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded, no extraneous text. Could be slightly more informative without being verbose, but acceptable for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a simple lookup with one parameter and no output schema, the description minimally covers the purpose. However, it lacks details on return format or examples, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (one parameter with a description). The description adds no additional meaning beyond the schema, but baseline is 3 for high coverage. No extra context provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'look up' and the resource 'payer-specific billing rules'. It distinguishes from sibling tools which focus on bundling, CPT, etc. However, 'billing rules' is somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. No context provided for appropriate use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_denialB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description should disclose behavioral traits like read-only or authorization needs, but it only mentions returning 'resolution steps', leaving safety and operational behavior unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no fluff, effectively front-loading the core function. It could be slightly more structured but is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description mentions 'resolution steps' but does not detail the output format, which is needed since there is no output schema. It is adequate but not comprehensive for a lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both 'code' and 'search' parameters. The description does not add additional meaning beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up denial codes (CARC/RARC) with resolution steps, specifying the verb 'look up' and the resource 'denial codes', and it distinguishes from sibling tools like lookup_bundling, lookup_cpt, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like lookup_cpt or lookup_payer. The description lacks context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_modifierB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only describes the purpose (lookup) without addressing permissions, rate limits, data freshness, or what constitutes 'usage' or 'documentation requirements'. The agent lacks insight into potential risks or constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence. While it efficiently conveys the core purpose, it could include more detail without becoming wordy. It earns a high score for lack of fluff but loses a point for being slightly under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no annotations), the description is incomplete. It does not hint at the return format, whether results are real-time, or what aspects of 'usage' are covered. A more complete description would improve agent confidence.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description of the 'modifier' parameter. The tool description adds no additional meaning beyond the schema. Baseline 3 is appropriate since the schema already documents the parameter adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Look up') and resource ('billing modifier usage and documentation requirements'). This distinguishes it from sibling tools that handle other billing concepts like bundling, CPT codes, denials, ICD10, and payers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus the siblings (e.g., 'for modifier-specific queries, use this; for code definitions, use lookup_cpt'). The intent is implied by the tool name and siblings, but a more direct statement would improve decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_cptA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It only states the lookup action without disclosing traits like read-only nature, authentication needs, rate limits, or whether it returns full code details. The agent is left to infer safe usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that efficiently conveys the tool's purpose with no extraneous information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with two well-documented parameters and no output schema, the description is fully adequate. It tells the agent what the tool does, how to invoke it, and the expected inputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The description adds value by implying the parameters are alternatives ('by code or search term'), which is not explicit in the schema. This helps the agent decide which parameter to use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'look up' and the resource 'CPT procedure codes', specifying the lookup method ('by code or search term'). It naturally differentiates from sibling tools like lookup_icd10 or lookup_modifier which handle different code types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for CPT code lookups, but provides no explicit guidance on when to use it versus sibling tools (e.g., lookup_icd10). No when-not-to-use or alternative suggestions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_icd10A
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. It implies a read-only lookup operation, which is adequate, but does not disclose any additional traits such as authentication requirements, result formatting, or rate limits. For a simple lookup, the minimal disclosure is acceptable but lacks richness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of ten words that immediately conveys the tool's purpose and input methods. No wasted words or redundancy. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description covers the essential use case: looking up codes by code or search term. It does not specify what the output contains (e.g., full diagnosis details, list of results), which is a minor gap, but the core functionality is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both parameters have descriptions). The description's phrase 'by code or search term' mirrors the schema but does not add new semantic value beyond what the schema already provides. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Look up' and the resource 'ICD-10 diagnosis codes', specifying two distinct input methods (by code or search term). This distinguishes it from sibling tools like lookup_cpt or lookup_modifier which deal with different code types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context of the sibling tools (all are lookup tools for different medical code types) implies this tool is appropriate when querying ICD-10 codes. No explicit exclusions or when-not-to-use guidance is provided, but the purpose is clear enough for basic selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.