Server Quality Checklist

Tool Scores

• get_decision_documentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden. It discloses it downloads PDFs from FDA servers and extracts text, and explains max_length controls output character count. It doesn't mention error cases or explicitly state read-only behavior, but the key behavioral traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized: a concise purpose sentence, a 'When to use' block, an Args list with format examples, and three concrete usage examples. Every sentence contributes useful information without redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters with format nuances and an output schema is present, so return structure need not be described. The description covers the workflow, parameter constraints, and examples. Minor gaps remain around error handling and document availability, but it's sufficiently complete for successful usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate and does so thoroughly. It provides exact formats for each document_type (e.g., 'K' + 6 digits for 510k_summary), clarifies supplement_number is required only for pma_supplement, and specifies max_length default (8000). This goes far beyond the bare string types in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Fetch FDA regulatory decision documents' and specifies it downloads PDFs and extracts text, clearly stating the action and resource. It distinguishes from sibling search_fda by emphasizing this tool retrieves full documents after a submission is found, not perform searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  An explicit 'When to use' section directs agents to use this after finding a device submission via search_fda, tying directly to sibling tool usage. It also notes the tool is 'not available via OpenFDA API,' implying a unique role, but it doesn't name exclusions for other siblings or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_searchable_fieldsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It transparently states the output ('field names, types, and descriptions') and the scope (all 21 endpoints, common vs. all fields). Although it doesn't explicitly say 'read-only', 'List' and 'Returns' make the non-mutating behavior clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every part earns its place: a one-sentence summary, a usage pointer, and a well-structured Args block. The endpoint list is necessary because the schema offers no descriptions. It is front-loaded and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, the presence of an output schema, and the lack of annotations, the description is complete. It fully covers the endpoint and category parameters, explains when to use the tool, and states what results to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It does: the 'Args' section enumerates every allowed endpoint path grouped by Drug/Device/Food/Other, and explains the category parameter ('common' default, 'all' complete listing). This is far more informative than the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List searchable fields for any OpenFDA endpoint. Returns field names, types, and descriptions.' This clearly distinguishes it from siblings like search_fda (which executes searches) and count_records (which counts records).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance: 'Call this BEFORE searching if you're unsure which field names to use in a search query. Field names vary between endpoints.' While it does not name a specific alternative tool, the 'before searching' naturally points to the search_fda sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_fdaA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that it returns individual records, not aggregates, and details query syntax with quoting and + for spaces, plus pagination and sorting behavior. It could mention rate limits or error handling, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but dense, with clear headers and practical examples. The dataset list is necessary given 21 enum values, and every section earns its place. Slightly verbose but not wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all essential aspects: purpose, usage guidance, parameter semantics, and examples. Since an output schema exists, not detailing return values is fine. This is a complete description for a complex tool with many dataset options.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds extensive meaning beyond the schema: dataset options are categorized and explained, search syntax is specified (quote strings, + for spaces), limit has default and max, skip is for pagination, and sort has a format example. This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it searches any of 21 OpenFDA datasets and returns individual records. It distinguishes from siblings by explicitly pointing to count_records for aggregation and list_searchable_fields for field guidance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit 'When to use' section, stating it's for finding specific records and directing users to count_records for statistics and list_searchable_fields if unsure of fields. This gives clear when-to-use and alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• count_recordsA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses return components (counts, percentages, narrative summary), the critical .exact suffix requirement with the consequence of tokenized counts, and the limit's default/max. It lacks rate limits or error behavior, but for this read/aggregate tool it provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a concise summary, 'When to use' section, labeled Args list, and practical examples. Every section earns its place and the markdown formatting improves scannability without adding fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 4 parameters and a non-obvious .exact suffix requirement, the description addresses the core nuances, gives multiple examples, and references sibling tools for additional context. The output schema exists, so not detailing return values is acceptable; the description still mentions the output composition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must fully compensate. It does: endpoint is explained with examples and a cross-reference, count_field gets a crucial .exact warning with text vs numeric/date distinction, search is described as an optional filter, and limit has default and max values. Examples further clarify usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Count/aggregate records by field across any OpenFDA endpoint,' giving a specific verb, resource, and scope. It distinguishes itself from the sibling tool search_fda, which is for individual records, and references list_searchable_fields for endpoint discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  An explicit 'When to use' section states to use this for statistics, distributions, or top-N lists, and instructs to use search_fda for individual records. It also cross-references list_searchable_fields, providing clear guidance on when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.