Server Quality Checklist

Tool Scores

• search_drug_labelsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It does state that the tool searches and returns labeling information including specific sections, which makes it clear this is a read-only operation and hints at output content. However, it omits details about result format, pagination/limit behavior, API constraints, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the action and purpose front-loaded ('Search FDA drug labels using OpenFDA API') and the return content summarized immediately. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has four optional parameters, all fully described in the schema, and the description states the output content, which makes it minimally viable. However, with no output schema and no annotations, the description could have been more complete by explaining pagination/limit interplay, clarifying the response structure, or distinguishing this from sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all four parameters with descriptions, defaults, and examples (100% schema description coverage), including clear examples for 'search' and 'count'. The tool description itself adds no additional parameter semantics beyond indicating the general search capability, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Search FDA drug labels using OpenFDA API') and lists the categories of returned information (indications, contraindications, warnings, adverse reactions). However, it does not explicitly differentiate this general label search from sibling tools like get_drug_warnings or get_drug_indications, which cover similar content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. Sibling tools such as get_drug_warnings, get_drug_indications, and get_drug_adverse_reactions exist but are not referenced, and there are no exclusion criteria or preferred contexts described.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_drug_adverse_reactionsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only restates the tool's function and source without detailing potential multiple records, matching behavior, pagination limits (beyond the schema), or any restrictions. This leaves the agent with insufficient information about how the tool actually behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and resource. Every word earns its place, with no superfluous information or repetition of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description should clarify the response format or behavioral nuances. It only says 'adverse reactions information' without specifying whether the response is a list, a structured object, or raw text, leaving the agent without adequate expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides complete descriptions for both parameters (100% coverage), so the description adds no additional meaning. It does not explain how drug_name is matched or what limit controls in terms of output, which is acceptable given the schema's thoroughness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('adverse reactions information'), and the specific scope ('for a specific drug from FDA labels'). This effectively distinguishes it from sibling tools like get_drug_warnings and get_drug_indications, which target different aspects of drug labels.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need adverse reactions for a drug) but provides no explicit guidance on when not to use it or how it compares to alternatives like search_drug_labels or get_drug_warnings. The intended use is clear from the context, but no exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_drug_indicationsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the high-level purpose and does not disclose that the tool can return multiple records (as suggested by the 'limit' parameter), the structure of the response, or any error behavior. The 'from FDA labels' detail adds source context but no operational transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that communicates the core purpose without any extraneous wording. It is concise and well-structured, making it easy for an agent to quickly grasp the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's apparent simplicity, the absence of an output schema and annotations places a higher burden on the description. The description does not explain that the tool returns potentially multiple records (the 'limit' parameter implies this), nor does it describe the return format or any limitations. This leaves gaps for an agent trying to predict the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both 'drug_name' and 'limit' already described in the schema. The tool description adds no additional parameter meaning beyond what the schema provides, so the baseline score of 3 applies. There is no missing parameter information that the description needs to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get indications and usage information for a specific drug from FDA labels.' The verb 'Get' is specific, the resource ('indications and usage information for a specific drug') is well-defined, and the source ('FDA labels') adds context. The tool is easily distinguishable from siblings like get_drug_adverse_reactions and get_drug_warnings, which target different data categories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when seeking indications/usage for a specific drug, but it does not explicitly state when to use this tool versus alternatives like search_drug_labels or ae_pipeline_rag. No exclusions or alternative recommendations are provided, leaving the usage context implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_drug_warningsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It only says 'Get', which implies a read operation, but does not mention pagination, return limits, or potential side effects. This leaves the agent without essential behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that communicates the core purpose without unnecessary words, making it highly efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description states the core function but does not explain the return structure or how 'limit' affects results. Since there is no output schema, the agent may be under-informed about response format, making it minimally sufficient but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no information about parameters beyond the schema, which already documents both 'drug_name' and 'limit' with 100% coverage. The schema handles parameter semantics, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the specific resource 'warnings and precautions' for a specific drug from FDA labels, which distinguishes it from sibling tools like get_drug_adverse_reactions and get_drug_indications.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case for a specific drug, but does not explicitly compare with alternative tools or provide exclusion criteria. It gives clear context without explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ae_pipeline_ragA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses the multi-step behavioral nature (fetch, extract, chunk, retrieve, summarize) and the intent to avoid truncation, but omits details like output format, potential external API calls, latency, or side effects. This is moderate disclosure, not rich enough for a 4 but above a 2.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two front-loaded sentences. The first states the core purpose, and the second elaborates with a clear list of actions and a rationale. Every clause adds value, with no fluff. It is an efficient, well-structured description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-step RAG, 5 parameters, no output schema), the description is incomplete. It does not specify the return format (summary vs. chunks vs. both), nor does it explain how the pipeline uses each parameter. An agent needs more detail to safely invoke and interpret results. The 'prevent truncation' note is helpful but insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add any parameter-specific meaning beyond the schema. It does not explain how parameters like 'drug', 'query', or 'filters' interact, so it neither enhances nor detracts from the schema's clarity. A score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is an 'Advanced RAG pipeline for drug safety analysis' that 'fetches, extracts, chunks, retrieves and summarizes FDA drug label data'. This specific verb list distinguishes it from the more focused sibling tools (e.g., get_drug_warnings) by promising an end-to-end analysis in one call, making its unique purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when a comprehensive, non-truncated analysis is needed ('in one call to prevent LLM response truncation'). It gives clear context for use but does not explicitly mention alternatives or exclusions, such as when to prefer a simpler sibling like get_drug_adverse_reactions. Thus it meets the 'clear context, no exclusions' criterion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.