Server Quality Checklist

Tool Scores

• get_europepmc_full_textC
  2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior but only notes 'without file saving'. It fails to state that the tool is read-only, what format the full text is returned in, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While the description is very short, it sacrifices essential information. Conciseness here leads to under-specification, making it less helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no output schema, and no annotations, the description is grossly inadequate. It does not explain return values, pagination (implied by offset/limit), or how to form requests.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no explanation for the three parameters (identifier, offset, limit). The agent has no clue what values to provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get full text without file saving' indicates the main action but lacks specificity about the resource (Europe PMC is implied by name). It does not differentiate from sibling tools like get_europepmc_paper_by_id or get_europepmc_pdf_as_markdown.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention any prerequisites, exclusions, or context for invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_identifiers_from_europepmcC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description needs to disclose behavioral traits. It only mentions 'without file saving,' but omits whether the operation is read-only, any side effects, authentication needs, or rate limits. This is insufficient for safe agentic use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at one sentence. While it avoids verbosity, it lacks critical information and does not effectively balance brevity with completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter and lack of output schema, the description should at least mention what the tool returns and any limitations. It fails to do so, making the tool underdocumented for effective selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one required parameter 'identifier' with no description. The tool description does not explain what 'identifier' means (e.g., DOI, PubMed ID), leaving the agent with no additional semantic context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Get all identifiers without file saving,' which indicates a retrieval action and clarifies that no files are saved. However, it does not specify what entity the identifiers belong to or differentiate from sibling tools like get_europepmc_full_text or search_europepmc_papers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance is provided. The description does not indicate when to use this tool over siblings, nor does it specify prerequisites or scenarios for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_europepmc_paper_by_idC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral transparency. It only adds 'without file saving', but does not disclose error handling, rate limits, authentication needs, or output structure. This is insufficient for a mutation-free read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence, which is concise. However, it omits necessary details, making it borderline under-specified rather than efficiently complete.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and annotations, and a single parameter, the description should cover accepted identifier formats and return structure. It does not, leading to significant gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the parameter 'identifier' has no description. The tool description adds no meaning beyond the parameter name, leaving the agent to guess what types of identifiers are accepted (e.g., PMID, DOI, or Europe PMC ID).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves Europe PMC paper metadata and explicitly notes 'without file saving', distinguishing it from some sibling tools. However, it does not fully differentiate from tools that also return metadata, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs. siblings like search_europepmc_papers, get_europepmc_full_text, or get_europepmc_pdf_as_markdown. The agent is left to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_europepmc_pdf_as_markdownC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It says 'without file saving', implying in-memory conversion, but doesn't mention network dependency, output format details, or error handling. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The single sentence is concise but overly terse. It conveys the main purpose but omits essential details, sacrificing completeness for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters (1 required), no output schema, and no annotations, the description is severely incomplete. It fails to explain input requirements, parameter usage, or return format, leaving the agent to guess.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description provides no explanation for any of the 5 parameters. 'identifier' is required but undefined; 'extract_tables', 'processing_method', 'offset', and 'limit' are completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Convert PDF to Markdown without file saving', which clearly identifies the action and resource. While it lacks explicit mention of Europe PMC, the tool name and sibling context imply that. It distinguishes from siblings like 'get_europepmc_full_text' (likely XML) and 'search_europepmc_papers' (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'get_europepmc_full_text'. No context on prerequisites for the identifier parameter or when conversion is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pmc_supplemental_materialA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided. The description does not disclose any behavioral traits such as side effects, permissions, rate limits, or error handling. It only describes the basic read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for supported formats, Args, Returns, and Examples. It is concise and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description adequately covers input parameters and return type. It lacks details on error cases but is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining each parameter (pmcid formats, idx as file index, offset and limit for windowing) and provides examples that clarify usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Gets Supplemental Material for a PubMed Central Open Access article.' It specifies the resource and action, and distinguishes from sibling tools that retrieve full text or identifiers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions supported input formats but provides no explicit guidance on when to use this tool versus siblings, nor any disqualifying conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_europepmc_papersA
  4.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description lacks behavioral details such as mutability, authentication, rate limits, or side effects. It only mentions that file saving is disabled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose, args/returns sections, and an example. It is front-loaded and concise without unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all three parameters, explains return type and structure, and includes an example. Despite no output schema, it provides sufficient context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides full parameter definitions including types, defaults, and options for result_type, which is absent from the input schema. Schema coverage is 0%, so description compensates entirely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Europe PMC for papers, with the specific caveat of not saving results to a file. This distinguishes it from siblings that focus on identifiers, full text, or PDFs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates it is for searching without file saving, but does not explicitly state when to use alternatives. However, sibling tools have distinct purposes, so context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.