Server Quality Checklist

Tool Scores

• fetch_articleA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the return values (title, abstract with section labels, authors, journal, etc.) and the error case (NOT_FOUND for invalid PMID). Missing any mention of side effects or limitations, but for a read-only fetch, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences. The first sentence states the core purpose, and the second enumerates output and error handling. No extraneous words, optimally front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 required parameter, output schema exists), the description covers the essential aspects: what it does, what it returns, and error handling. It could mention input format expectations more explicitly, but the example in schema suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a clear description and example for the 'pmid' parameter. The main description adds no further semantics beyond stating the tool's purpose. Baseline score is appropriate as the schema already documents the parameter well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch one article with full abstract text', specifying the verb 'fetch' and the resource 'one article'. It lists the output fields, distinguishing it from sibling tools that search or find related articles. This is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a PMID is available through the required parameter, but does not explicitly state when to use this tool versus alternatives like search_pubmed or find_related. No guidance on when not to use or for broader queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_relatedA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the algorithm (probabilistic topic-similarity model) and return type (slim metadata), but does not explicitly state that the operation is read-only or without side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, front-loaded with the core action, and contains no redundant or extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema covering return values, the description provides sufficient context for understanding the tool's purpose and behavior, including its algorithm and typical use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters. The description adds minimal new parameter-specific info, only indirectly describing the purpose of the pmid parameter through the tool's action.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Walk the PubMed citation neighbor graph') and resource ('from one PMID'), clearly distinguishing it from sibling tools like fetch_article (single article) and search_pubmed (general search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states a use case ('what else should I read about this paper?' follow-ups). It does not explicitly mention when not to use it or alternative tools, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_pubmedA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It reveals the two-step process (esearch + esummary) and the specific metadata fields returned (title, authors, journal, pubdate, DOI). It does not disclose rate limits, authentication needs, or whether the tool is read-only, but for a public database search, the implied read-only nature is acceptable. The description adds value beyond annotations (which are absent) by explaining internal behavior and output scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each with a distinct purpose: purpose, internal process, next-step advice. No wasted words, front-loaded with the core action. Every sentence is informative and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown here but flagged as present), the description does not need to detail return structure. It already lists the key fields. The tool has three parameters, all documented in schema, and the description covers the overall behavior, internal steps, and follow-up tool. This is complete for a search tool with clear sibling relationships.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: all three parameters have clear descriptions in the input schema. The tool description does not add further detail about parameter meaning (e.g., accepted query syntax is mentioned only in schema, not in description). Baseline is 3 because the schema already documents parameters well; the description provides no additional semantic value for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search PubMed and return slim metadata for each hit', which is a specific verb-resource combination. It distinguishes from the sibling 'fetch_article' by explicitly noting that tool is for full abstracts. The two-step internal process (esearch + esummary) is also mentioned, providing clarity on what 'slim metadata' means.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use the sibling 'fetch_article' ('Use fetch_article next when you need the full abstract'). This helps the agent choose the correct tool. However, it does not address other siblings like 'find_related' or 'search_by_author', leaving some gaps in when-not-to-use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_authorA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully covers behavioral traits: it wraps the underlying search with a field qualifier, sorts by newest-first by default, and allows sorting by relevance. No contradictions are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero wasted text. Each part serves a purpose: stating the core function, explaining the wrapping mechanism, stating default behavior, and offering an actionable alternative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, an output schema exists, and the description covers all essential aspects: purpose, behavior, sorting options, and differentiation from siblings. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the default sort ('Newest-first') and the effect of the 'relevance' option ('surface most-cited / highest-impact first'), enhancing understanding beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and a clear resource ('author's PubMed-indexed publications'). It distinguishes from the sibling 'search_pubmed' by explaining it wraps with the '[au]' qualifier, making the tool's unique purpose explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: use this tool for author-specific searches to avoid partial-name matches, and mentions the default sort order and the option to switch to 'relevance'. It implies 'search_pubmed' is for broader queries, giving clear when-to-use and when-not-to-use information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.