Server Quality Checklist

Tool Scores

• pubmed_extract_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint, covering safety and side-effect expectations. The description adds context about token efficiency and the targeted nature of extraction, but does not disclose return format or further behavioral details. This is adequate but not rich beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, then usage guidance. No filler or redundancy. Every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity (2 params, no output schema, read-only), the description provides enough context: what it extracts, when to use, and how it compares to a sibling. It does not explain return values, but the annotations cover safety and the description conveys the core use case well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: both 'pmid' and 'sections' have descriptive text in the schema. The description mentions examples of sections (authors, abstract, keywords, DOI) which echo the schema enums, adding no new meaning. Baseline 3 is appropriate since the schema handles parameter semantics fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Extract specific structured sections from a PubMed article.' It uses a specific verb ('extract') and resource ('sections from a PubMed article'), and differentiates from siblings by noting it is for partial data rather than the full record, explicitly contrasting with get_details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use: 'Use when you need only certain aspects… rather than the full record.' It also names the alternative (get_details) and highlights token efficiency. It does not explicitly state when not to use, but the 'when' scenario is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pubmed_find_relatedA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint, covering the safety/behavioral profile. The description adds context about discovering similar papers or reviews but does not disclose additional behaviors such as handling of invalid PMIDs, pagination, or response format. Given the annotations, the description meets baseline but adds limited new behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences with no redundant words. The first sentence front-loads the core purpose, and the second provides actionable usage context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple tool with well-documented schema and annotations, the description provides the essential purpose and guidance. No output schema exists, but the description's mention of 'Find articles related' implies the return of a list of related articles. It lacks explicit details about output format or edge cases, but given the tool's simplicity, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed descriptions for all parameters. The description adds no meaningful parameter semantics beyond restating what the schema already provides (e.g., 'similar papers or review articles' mirrors the 'type' enum). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Find articles related to a given PubMed article.' It further distinguishes from siblings by mentioning 'similar papers or review articles' and the workflow of expanding a search after identifying a key paper. This clearly differentiates it from pubmed_search, pubmed_get_details, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: 'Use after identifying a key paper to expand your search in relevant directions.' This implies a workflow and distinguishes when to use this tool relative to an initial search. However, it does not explicitly name alternatives or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pubmed_searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and openWorldHint, so safety is covered. The description adds value by disclosing that results include metadata, abstracts, and citations, and by mentioning supported query features like Boolean operators and MeSH terms. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, front-loaded with the core purpose, and every sentence provides useful information without redundancy. It is compact and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main output contents and several key parameters, and the schema covers the remaining parameters. With no output schema, the description explains that results contain metadata, abstracts, and citations. It could arguably mention sort_by or max_results usage, but it is otherwise complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, providing a baseline of 3. The description adds meaningful extra detail beyond the schema for query, mentioning field tags like [Title] and [Author], and gives usage guidance for format and days_back, which helps the agent choose parameter values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search PubMed biomedical literature database' and describes the output as 'articles matching your query with metadata, abstracts, and citations.' This specific verb+resource phrasing distinguishes it from sibling tools like pubmed_get_details or pubmed_find_related.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear in-tool guidance for format and days_back, but it does not explicitly say when to use this tool versus alternatives like pubmed_get_details or pubmed_find_related. Usage guidance is mostly implied by the search-oriented purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pubmed_get_detailsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, open-world, and idempotent behavior. The description adds the limit of up to 20 PMIDs and the specific fields returned (abstract, authors, journal, DOI, MeSH terms), which is useful context beyond the annotations. It doesn't mention error handling, but the annotation coverage lowers the burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main action, and contains no filler or redundant information. Every sentence provides value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, with 2 parameters and annotations but no output schema, the description covers the input format, limits, expected return content, and usage context. This is sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description only restates what the schema already says (single or array, up to 20), adding no additional parameter-specific meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve complete metadata for specific PubMed articles by PMID', which uses a specific verb and resource. It distinguishes from siblings by focusing on PMID-based retrieval rather than searching, and mentions 'complete metadata' which is distinct from other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It says 'Use when you have specific PMIDs from a previous search or citation and need the full record', providing clear context. However, it doesn't explicitly name alternative tools (e.g., pubmed_search) or state when not to use it, so it falls short of full explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pubmed_manage_cacheA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true. The description adds valuable nuance by showing that 'stats' is safe, 'clean' removes expired entries (moderate), and 'clear' wipes a cache layer (destructive). It clarifies the specific destructive behavior for each action, which is more granular than the single annotation flag.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, followed by concise action-specific guidance. Every word earns its place – no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 enum params, no output schema, annotations present), the description covers the key behavioral aspects: the three actions, their intent, and the existence of cache layers via 'target'. It could mention that 'clear' is irreversible or what 'stats' returns, but the current description is sufficient for basic correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with enum parameters already explaining each field. The description adds semantics by mapping actions to real-world outcomes (checking hit rates, removing expired entries, wiping layers), which deepens the meaning beyond the generic 'Cache operation to perform.' This extra context helps the agent choose the right action without additional lookups.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'View cache statistics or perform maintenance,' clearly stating the tool's purpose as cache management. It explicitly distinguishes itself from sibling tools (search, details, extract, related) by focusing on internal DB cache operations. The actions are enumerated with specific outcomes (hit rates, storage usage, expired entries, wipe layer).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by explaining when to use each action: 'stats' for hit rates and storage, 'clean' for removing expired entries, 'clear' for wiping a layer. While it doesn't explicitly mention alternatives to this tool, the sibling tools are obviously unrelated to cache management, making the intended use case clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.