Server Quality Checklist

Tool Scores

• niaid_data_queryA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark it as read-only, idempotent, and non-destructive. The description adds substantial behavioral context: it calls an external API with rate limiting and timeout errors, returns formatted output (markdown or JSON), and includes pagination and aggregation details beyond the annotation hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-organized into Purpose, API context, Args, Returns, Examples, and Error Handling. It front-loads the core purpose in the first sentence, and every section adds operational value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description includes the full JSON response schema, markdown output structure, error handling, and multiple examples. Even with the output schema present, it explains expected behaviors thoroughly, making it complete for a complex query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description's Args section goes beyond the schema by adding Elasticsearch syntax examples ('name:COVID', '@type:Dataset'), explaining how 'offset' and 'size' work together for pagination, and clarifying that 'size=0' can be used with aggregations. The examples show concrete parameter combinations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Search the NIAID Data Ecosystem for biomedical research resources,' clearly stating the verb and resource. It further explains it queries the API and supports Elasticsearch syntax, but it does not explicitly contrast with sibling tools like niaid_data_get or niaid_data_summarize.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Examples' section provides multiple 'Use when:' scenarios, giving explicit context for when to invoke the tool, such as 'Find open-access COVID-19 datasets' and 'What types of resources are available?'. However, it does not mention when not to use it or suggest alternatives for single-record retrieval or other sibling operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• niaid_data_getA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds value by explaining the two response formats (json and markdown), the complete set of fields returned, and the specific error messages for not-found and API failures. This goes beyond annotation-only information, though it does not disclose potential performance or authentication nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized into clear sections: a one-sentence overview, args, returns, error responses, and examples. It is front-loaded with the purpose and every section adds practical value. No wasted words or redundant repetition of schema or annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the relative simplicity of the tool, the description covers all necessary context: what the tool does, when to use it in the broader workflow, parameter usage, return format details, error behaviors, and concrete examples. With an output schema implied and annotations present, no further details are needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already describes both parameters at 100% coverage, the description significantly enriches them. It clarifies that the id comes from the '_id' field of query results and provides a concrete example. It also explains the difference between 'json' and 'markdown' response formats and their defaults, adding meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Fetch the complete record for a single NIAID Data Ecosystem resource by its ID', a specific verb+resource+method statement. It further distinguishes itself from siblings by explicitly noting 'Use this after niaid_data_query to get the full details of a record of interest.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clear guidance is provided: 'Use this after niaid_data_query to get the full details of a record of interest.' This names the alternative query tool and indicates the workflow. Examples also show exactly when to use the tool, satisfying the explicit 'when' and 'alternative' requirements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• niaid_data_summarizeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although annotations already declare readOnlyHint, openWorldHint, and idempotentHint, the description adds meaningful behavioral context beyond them: it does NOT return all records, returns a compact overview, limits sample records to sample_size, and has no side effects. No contradiction exists between annotations and description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but extremely well-structured with clear sections: overview, when to use, args, returns, and examples. It is front-loaded with the core purpose, and every section adds actionable information without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is fully self-contained: it explains the tool's exact output structure, parameter behavior, examples, and exclusions. Given the tool's moderate complexity and rich input schema, this description provides complete guidance for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers both parameters with detailed descriptions, giving a strong baseline. The tool description reinforces key semantics ('same syntax as niaid_data_query', 'Use * for the entire catalog', 'Does NOT return all results') and adds practical examples, though much of this duplicates the schema text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Summarize a NIAID Data search without returning all records to the context.' It clearly distinguishes this from the sibling tools niaid_data_query and niaid_data_get by stating it provides an overview rather than full records.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance with a bulleted list ('Use this tool BEFORE niaid_data_query when a search may return many results') and explicit when-not-to-use guidance ('Don't use when: You need specific records (use niaid_data_query instead)'). This fully clarifies tool selection versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.