Server Quality Checklist

Tool Scores

• format_bibtexB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden of explaining behavior. It states that a standardized BibTeX entry is generated, but it does not disclose output format, citation-key behavior, handling of missing optional fields, or any validation/error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It communicates the core action and the key inputs immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a straightforward formatting tool, and the schema covers all parameters. However, with no output schema and no annotations, it would benefit from describing the returned BibTeX format and clarifying optional DOI behavior, so an agent is not left guessing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes all parameters, so the baseline is 3, but the description introduces ambiguity by listing DOI as an input even though it is optional in the schema, while omitting URL entirely. This can mislead the agent into treating DOI as required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Generates') and resource ('standardized BibTeX entry'), and lists the required inputs. This clearly distinguishes it from sibling tools like verify_citation and search_academic_papers, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus the alternatives. The description implies use for generating BibTeX, but it never mentions conditions, exclusions, or how it relates to verify_citation or search_academic_papers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_academic_papersA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It reveals that multiple databases are queried, which is useful, but it does not disclose key behaviors an agent would care about: return format, result limits, pagination, deduplication across sources, or failure/partial-failure behavior when one database is unreachable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is efficiently front-loaded with the core action and scope, followed by the source list. No filler or redundancy; every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only 2 parameters and full schema coverage, the description covers the input side adequately. However, there is no output schema and the description never mentions what the tool returns (e.g., papers with titles, authors, years, relevance ordering), leaving the agent guessing about the result shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters well. The description adds marginal value by reinforcing that 'query' can be a topic, keywords, or research question and that 'source' maps to the listed databases, but it does not go beyond the schema's existing detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses a specific verb ('Searches') with a concrete resource ('top-tier academic databases') and even names the exact databases (Crossref, Semantic Scholar, DBLP, arXiv). An agent can immediately understand what the tool does without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use case is implied by the description ('papers matching a topic, keywords, or research query'), but there is no explicit guidance on when to prefer this tool over alternatives, nor any stated exclusions. With no sibling tools provided, some implied usage is acceptable, but the guidance remains implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_citationA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It does well by revealing that it queries Crossref, Semantic Scholar, DBLP, and arXiv concurrently, computes confidence scores, resolves DOIs, and returns formatted citations. It stops short of describing failure/not-found behavior or the exact shape of the verification result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense sentence with no filler; the core purpose is front-loaded and every clause adds useful behavioral detail. It is slightly run-on and could be split into separate sentences for readability, but it remains efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool does and its major outputs, but no output schema exists, and the description does not clarify the result structure, how confidence scores are represented, or what happens when a citation cannot be verified. Given the multi-source external lookups, more detail would help an agent handle partial or failed lookups.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters adequately. The description adds no significant parameter-level semantics beyond mentioning DOI resolution, which is already implied by the 'doi' parameter description. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('verifies whether') and a specific resource ('academic paper citation'), and clearly distinguishes the tool from its siblings: it checks authenticity rather than searching for papers or merely formatting citations. The phrase 'authentic or AI-hallucinated' adds precise functional intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies clear usage context: use this tool when a citation needs validation. It also implicitly distinguishes from search_academic_papers and format_bibtex by focusing on verification and multi-source lookup. However, it does not explicitly state when not to use it or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.