Server Quality Checklist

Tool Scores

• search_pubmedA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and succeeds by detailing the return format as 'a formatted list of matching articles with PMID, title, authors, journal, date, and a short abstract snippet.' It also discloses error handling behavior, noting it returns an error message if the query fails or yields no results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description uses a clear structured format with Args and Returns sections, and every line provides valuable information such as PubMed syntax examples and field tags. While lengthy, the examples are essential given the complexity of PubMed search syntax, making the length appropriate rather than wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero percent schema coverage and lack of annotations, the description is remarkably complete by documenting all input parameters with examples and explaining the output structure in the Returns section. No significant gaps remain for an agent to successfully invoke this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description comprehensively documents all six parameters through the Args section, including syntax examples for the query parameter, valid ranges for max_results, and allowed values for article_type and sort. This fully compensates for the complete lack of schema metadata.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Search PubMed for articles matching a query,' providing a specific verb and resource. It implicitly distinguishes itself from sibling tools like get_article (which likely retrieves by ID) and search_by_author by emphasizing query-based matching rather than specific ID retrieval or author searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description explains how to construct queries using PubMed syntax, it lacks explicit guidance on when to use this tool versus siblings like search_by_author. The Args section implies usage through examples but does not explicitly state when to choose this over alternative search methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_related_articlesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and discloses the specific algorithm used, return format ('ranked list with brief metadata'), and error conditions ('Returns an error message if the PMID is invalid'). It could be improved by explicitly stating the read-only/safe nature of the operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with clear sections: purpose statement, algorithm explanation, Args block, and Returns block. Every sentence provides unique value without repetition, and critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 simple parameters and no complex nested structures, the description is complete: it covers the algorithmic approach, parameter semantics, return value structure, and error handling. The presence of return value documentation in the description compensates adequately despite the lack of formal output schema in the structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage (schema only has titles 'Pmid' and 'Max Results'), the description fully compensates by documenting both parameters in the Args section: pmid is defined as 'The PubMed ID of the reference article' and max_results includes constraints '1-50, default 10' and semantic meaning 'Number of related articles to return'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Find') + resource ('PubMed articles') + relationship ('related to a given article'), and further distinguishes itself from siblings by specifying it uses NCBI's 'similar articles' algorithm based on co-citation and text similarity, clearly differentiating it from general search tools like search_pubmed or retrieval tools like get_article.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the algorithm explanation (co-citation/text similarity), suggesting when to use this versus keyword searching, but lacks explicit when-to-use/when-not-to-use guidance or direct comparison to sibling alternatives like search_pubmed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_articleA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and successfully discloses error behavior ('Returns an error message if the PMID is invalid or not found'). Lacks mention of rate limits or authentication requirements, but covers the critical failure mode.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses a structured docstring format (Args/Returns) with zero waste. The first sentence establishes purpose, followed by precise parameter documentation and return value specification. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter lookup tool, the description is comprehensive: it documents the input parameter (compensating for schema gaps), lists return fields, and specifies error behavior. Presence of output schema reduces the need for detailed return documentation, yet it provides helpful field enumeration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (parameter has no description field), but the description fully compensates by documenting the PMID parameter with semantics ('PubMed ID'), format hint ('numeric string'), and a concrete example ('33982811').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with specific verb ('Get'), resource ('PubMed article'), and scope ('complete details by its PMID'), clearly distinguishing it from sibling search tools and full-text retrieval tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'by its PMID' provides clear context that this tool requires a specific identifier, implicitly distinguishing it from search_pubmed and search_by_author siblings. However, it does not explicitly name alternatives or state when to use search vs. retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_full_textA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and successfully discloses dual-output behavior (full text with structure 'title, abstract, and body sections' vs 'helpful message with links'), and the PMC open-access constraint. Lacks mention of rate limits, caching, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses standard docstring format (Args/Returns) that is appropriate for the documentation burden. Every sentence serves a distinct purpose: capability statement, constraint warning, fallback behavior, parameter definition, and return value specification. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with conditional output logic, the description is comprehensive. It explains the parameter (since schema fails to), documents both success and failure response patterns, and clarifies content scope (PMC open-access only) without needing to duplicate the output schema structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Compensates effectively for 0% schema description coverage by defining the 'pmid' parameter as 'The PubMed ID of the article,' providing essential semantic mapping. Would benefit from format hints (e.g., numeric string) or validation constraints to achieve a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Retrieve') and resource ('full text of an article from PubMed Central'), clearly distinguishing it from siblings like 'get_article' (likely metadata) and 'search_pubmed' (discovery). The 'if available' qualifier immediately signals the conditional nature of this operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly defines scope limitations ('Only open-access articles archived in PMC') and failure mode behavior ('Subscription-only articles will return a link'), guiding users on when the tool will and won't return full text. Lacks explicit cross-reference to sibling tools (e.g., 'use get_article for metadata-only retrieval').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_authorA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully discloses sorting behavior ('sorted by most recent first') and error conditions ('Returns an error message if the author name is empty'). Lacks only details on pagination or rate limiting.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Efficiently structured with clear Args/Returns sections. Every element serves a purpose: examples clarify PubMed naming conventions, ranges prevent invalid inputs, and the Returns section sets expectations. No redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive for a 4-parameter search tool. Despite existing output schema (not shown), the description adds essential behavioral context (sorting order, error formats) that schemas typically don't convey. Fully covers all parameters given the zero schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Excellent compensation for 0% schema coverage. The Args section provides detailed semantics: three concrete format examples for 'author' (Smith JA, Smith J, John Smith), valid range for 'max_results' (1-100, default 10), and clear temporal semantics for year filters ('from this year', 'up to this year').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with specific verb+resource: 'Search PubMed for all articles by a specific author.' The phrase 'by a specific author' clearly distinguishes this from the sibling tool 'search_pubmed' (general search) and positions it as the correct choice for author-specific queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context through the specificity of 'by a specific author,' implying the exact use case (author-based retrieval). However, it lacks explicit 'when not to use' guidance or explicit comparison to 'search_pubmed' for broader queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.