Server Quality Checklist

Tool Scores

• get_fulltextC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It mentions resolving to latest version by default, which is a behavioral detail, but does not disclose other traits like idempotency, authentication needs, rate limits, or error semantics. As a read-like tool, more transparency about what exactly is returned is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the primary action. No unnecessary words, but lacks structured formatting (e.g., bullet points) which could improve scanability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, no output schema, and no annotations, the description is too brief. It does not explain the structure of returned full-text sections, error behaviors, or boundary conditions (e.g., invalid DOI). Agents would need to infer behavior from the name and schema alone, which is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It partially covers 'doi' and 'version' but does not mention 'as_markdown' at all. The description also says 'or URL' but the schema only requires a doi string, which could be misleading. Only 2 of 3 parameters are addressed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves structured full-text sections for a preprint identifier (DOI or URL). It uses a specific verb and resource, distinguishing it from siblings like search_preprints (search) and get_metadata (metadata). However, it does not explicitly contrast with resolve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus siblings (search_preprints, get_metadata, resolve). It mentions version parameter handling but lacks context about prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolveB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It only says 'resolve between' without explaining return values, side effects, or behavior on failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words, front-loaded with core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema and description does not specify return format or completeness of resolution. Incomplete for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% parameter description coverage. Description adds that the parameter can be a DOI or OpenAlex id, but is ambiguous about PPR id inclusion; the parameter name contradicts the description (doesn't mention PPR).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool resolves between three specific identifier types (DOI, OpenAlex id, Europe PMC PPR id), distinguishing it from sibling tools (search, get metadata, get fulltext).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. It's implied for identifier conversion, but no when-not or prerequisite info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metadataB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description must disclose all behavioral traits. It notes 'abstract only' but omits read-only nature, error handling (e.g., invalid DOI), authentication needs, or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no redundant words, front-loaded with verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is adequate but lacks completeness: missing information on return format, error responses, or usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds that 'doi' is a 'preprint DOI', but with 0% schema coverage, more detail (e.g., format, examples) is expected to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear verb ('Retrieve') and resource ('bibliographic metadata (abstract only)') for a 'preprint DOI', distinguishing it from siblings like search_preprints (searching) and get_fulltext (full text).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. Does not mention when not to use or connect to sibling tools like get_fulltext for full text.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_preprintsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behaviors. It mentions relevance/ngram sorting but fails to mention rate limits, result format, pagination, or destructive nature. 'Search' implies read-only but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence packs purpose, sources, and field scopes without redundancy. Could be slightly clearer by splitting metadata vs. behavior, but remains efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 4 parameters and no output schema or annotations, the description omits critical details: pagination, ordering, result structure, error handling. Incomplete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, description adds meaning to 'field' (scopes) and 'source' (Europe PMC, OpenAlex). But 'limit' and 'query' lack explanation; query syntax, case sensitivity, or formatting not described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches preprint full text, specifies two relevance methods (Europe PMC, OpenAlex n-gram), and lists field scopes. This distinguishes it from siblings like get_fulltext or get_metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context for search (preprint full text) but no explicit guidance on when to use this tool versus alternatives like get_fulltext or resolve, nor any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.