Server Quality Checklist

Tool Scores

• ae_pipeline_ragC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It lists pipeline steps (fetch, extract, chunk, retrieve, summarize) but does not mention output format, external API calls, rate limits, permissions, or potential side effects. The truncation rationale is vague and unhelpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact at two sentences. The first sentence delivers the core purpose; the second adds context, though it is slightly awkward. Overall, it is appropriately sized without excessive verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool with 5 parameters, a nested filters object, no output schema, and no annotations. The description omits essential details such as return value structure, expected latency, and when to prefer this over simpler sibling tools, making the overall picture incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and each parameter has a self-explanatory description, so the baseline of 3 applies. The tool description adds no additional context about parameter interactions, defaults, or precedence beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: a RAG pipeline that fetches, extracts, chunks, retrieves, and summarizes FDA drug label data, distinguishing itself from sibling tools that focus on individual aspects. However, the phrase 'Advanced RAG pipeline' is technical jargon and 'to prevent LLM response truncation' is ambiguous, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is given. The phrase 'in one call' implies it replaces multiple separate calls, but no alternatives are named and no exclusion criteria are provided, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_drug_warningsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the data source ('FDA labels') without explaining return behavior, pagination, the effect of the 'limit' parameter, or any potential side effects. This is minimal and lacks needed context for an agent to anticipate tool behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no unnecessary words. It is front-loaded with the verb and resource, making it quick to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, and the description does not explain what a warning record looks like, whether the result is a list, or how the limit parameter affects results. While parameters are well-documented, the lack of return-value context makes the description incomplete for an agent to understand the full tool behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% with clear descriptions for both 'drug_name' and 'limit'. The tool description itself does not add parameter-specific meaning beyond the schema, but the schema already provides sufficient semantics, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('warnings and precautions for a specific drug from FDA labels'), clearly stating the tool's function. It does not explicitly distinguish from sibling tools like 'get_drug_adverse_reactions' or 'get_drug_indications', but the resource is distinct enough to infer the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention situations where other tools (e.g., search_drug_labels or get_drug_adverse_reactions) might be more appropriate, nor does it state any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_drug_adverse_reactionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden. It only mentions the data source (FDA labels) but does not describe the return format, handling of missing drugs, pagination behavior, or any constraints. The 'get' verb implies read-only but adds no deeper behavioral insights.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that directly states the purpose without any filler. It is front-loaded with the verb and resource, making it easy to scan and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations or output schema, the description is minimal and leaves out important behavioral details, such as what the returned 'adverse reactions information' looks like, how many records are returned, or how it compares to sibling tools. For a simple getter it is minimally sufficient but clearly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are already fully described in the schema (100% coverage). The description adds context by noting 'specific drug' and 'from FDA labels', but it does not clarify the behavior of the limit parameter beyond what the schema already states. Baseline 3 is appropriate since the schema carries the detailed parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action (get), the resource (adverse reactions information), the specific target (a specific drug), and the source (FDA labels). It naturally differentiates from sibling tools like get_drug_warnings and get_drug_indications.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as search_drug_labels or ae_pipeline_rag. The description only states what it does, not the preferred context or scenarios to avoid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_drug_labelsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose rate limits, authentication requirements, pagination behavior, or whether the operation is read-only. The return format is only vaguely implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, information-dense sentence that front-loads the primary purpose and expected return. No wasted words or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description should provide more behavioral and usage context. It does not explain how search syntax works, what the response structure looks like, or when to prefer sibling tools. The 4-parameter tool needs more guidance for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all 4 parameters, including examples. The description adds no additional parameter semantics, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search FDA drug labels') and clearly states the resource and return content (indications, contraindications, warnings, adverse reactions). This distinguishes it from sibling tools that target specific sections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a general search use case but provides no explicit guidance on when to use this vs. sibling tools like get_drug_adverse_reactions or get_drug_warnings. No exclusions or alternative recommendations are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_drug_indicationsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It clearly states the data source (FDA labels) and the type of content returned (indications and usage), but it does not explain behavior around the 'limit' parameter, error handling, or response format. This is adequate for a simple read operation but not rich in detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that conveys the essential purpose without any redundant words. It is well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter read operation, the description is largely complete: it states the action, the resource, and the data source. It does not specify the response shape or the effect of 'limit', but given the absence of an output schema and the basic nature of the tool, this is a minor gap rather than a critical omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, as both 'drug_name' and 'limit' have descriptive inline comments. The tool description aligns with the primary parameter 'drug_name' but adds no extra meaning beyond what the schema already provides. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with the resource 'indications and usage information' for 'a specific drug', clearly defining the tool's purpose. It naturally distinguishes itself from sibling tools like get_drug_adverse_reactions and get_drug_warnings by focusing on indications/usage rather than adverse events or warnings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a specific drug' implies that the tool is intended for precise lookups when a drug name is known, contrasting with broader search tools like search_drug_labels. However, it lacks explicit statements about when not to use this tool or explicit naming of alternatives, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.