Server Quality Checklist

Tool Scores

• medlineplus_connectC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It only states it is a 'lookup', implying a read-only operation, but lacks detail on authentication, rate limits, or any potential side effects. The brevity leaves ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, making it concise. It is front-loaded with the essential verb and object. However, it could benefit from a bit more detail without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 parameters and no output schema, the description should provide more context about the lookup process and expected results. It is too minimal to fully inform an agent about how to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 40% (language and codeSystem have descriptions). The description adds no additional meaning to the parameters; it merely echoes 'type/code-system combinations' without clarifying the roles of 'code' and 'name' or how they interact with 'type' and 'codeSystem'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs a low-level lookup for supported type/code-system combinations. This distinguishes it from the more specific sibling tools (lookup_medication, etc.) by emphasizing its generic nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus the more specific siblings. The description does not specify prerequisites, when not to use it, or provide any explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_lab_testA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose any behavioral traits such as idempotency, side effects, rate limits, prerequisites, or error handling. The description carries the full burden but adds no behavioral context beyond the basic lookup action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the tool's purpose. Every word is necessary, and there is no redundancy or trivial content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description lacks information about the return format (e.g., plain text, structured data, links). Although the task is simple, the description could be more complete by clarifying what the output contains. The current version is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions. The description adds no additional meaning beyond the schema; it only mentions using a LOINC code. Since schema coverage is high, baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'look up', the resource 'MedlinePlus lab-test information', and the method 'using a LOINC code'. It distinguishes this tool from siblings like lookup_medication, lookup_diagnosis, lookup_procedure, and medlineplus_connect by specifying the type of information and the required identifier.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is provided. The description implies that this tool is for lab tests with LOINC codes, but does not mention alternatives or situations where other siblings might be better. Usage context is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_procedureA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It does not mention that the tool is read-only, any authentication requirements, or the nature of the output (e.g., what information is returned). This is insufficient for safe agent invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no superfluous words. It is front-loaded with the action and resource. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and sparse annotations, the description does not explain what the tool returns (e.g., full MedlinePlus entry, summary, links) or any limitations (e.g., rate limiting, required codes). A lookup tool with only input specs is incomplete for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides descriptions for 3 of 4 parameters (75% coverage). The description adds context about MedlinePlus and code systems but does not elaborate on parameter semantics beyond what the schema states. No additional format or syntax guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Look up'), the resource ('MedlinePlus information'), and the specific input types ('CPT or SNOMED CT procedure codes'). It distinguishes itself from sibling tools (lookup_medication, lookup_diagnosis, etc.) by focusing on procedure codes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for procedure code lookups via the input schema and mentions of CPT/SNOMED CT. However, it lacks explicit guidance on when to use this tool over siblings or specific prerequisites (e.g., requiring an authenticated session).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_medicationA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses source (MedlinePlus Connect) and acceptable inputs, but lacks details on failure modes, data freshness, or behavior if multiple identifiers are provided. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence conveys purpose, source, and input methods with no wasted words. Front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, yet description doesn't explain return format or provide examples. Lacks details on error handling or network requirements. For a simple lookup, it's adequate but leaves gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 4 params with descriptions (100% coverage). Description adds context: 'English name fallback' and that the tool uses codes or name. This extra clarity about fallback behavior justifies above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'look up', resource 'medication education links from MedlinePlus Connect', and acceptable identifiers (RXCUI, NDC, name). Distinguishes from sibling tools (lookup_diagnosis, etc.) by focusing solely on medications.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for medication lookups but offers no explicit guidance on when to use vs alternatives, when not to use, or prerequisites. The purpose alone differentiates from siblings, but no exclusions or decision tree.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_diagnosisA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('look up') and resource ('MedlinePlus health-topic links'), but does not disclose what happens if the code is not found, any rate limits, or the structure of the returned links. It is minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-formed sentence that conveys the core purpose without extraneous words. It is front-loaded with the action and specific code systems, making it efficient for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description could be more explicit about the return format (e.g., list of links, titles). However, for a straightforward lookup tool, the mention of 'health-topic links' provides sufficient context for an agent to understand the general outcome. It is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75%, meaning the schema already documents most parameters well. The description adds context that the 'codeSystem' parameter includes ICD-10-CM, ICD-9-CM, or SNOMED CT, which aligns with the schema enum. However, it does not significantly enhance understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it looks up MedlinePlus health-topic links for diagnosis codes, specifying three code systems (ICD-10-CM, ICD-9-CM, SNOMED CT). This distinguishes it from sibling tools like lookup_medication or lookup_lab_test, which target different medical entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While no explicit 'when to use' or 'when not to use' guidance is given, the tool name and description make it obvious that it is intended for diagnosis codes. The context of sibling tools implies alternative tools for medications, labs, procedures, and general MedlinePlus connections, providing indirect usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.