Server Quality Checklist

Tool Scores

• search_papersB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It mentions combining results from two databases but lacks disclosure on side effects, rate limits, error handling, or authentication requirements. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with a brief sentence and an Args section. Mixed language (German/English) but still efficient. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema; description covers purpose, parameters, and basic behavior. However, it does not describe the return format or fields. For a simple search tool, it is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description adds value by explaining that 'query' is search terms and 'limit' is max results per source with default (10) and max (50). However, it omits details like query syntax (boolean, phrases) and whether 'limit' is per source or total.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches for scientific papers via Crossref and OpenAlex, combining results. It distinguishes from siblings like get_author_profile, get_citations, get_paper_details, and search_topics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Says 'Ideal für Literaturrecherche' but does not explicitly state when to use vs alternatives or provide exclusions. Usage guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_citationsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist. The description mentions Semantic Scholar but fails to disclose important behavioral traits such as rate limits, auth requirements, or the exact return structure (e.g., pagination).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two short paragraphs plus Args) and front-loaded with purpose. The Args section is clear, though the overall structure could be slightly more standardized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and low schema coverage, the description provides adequate context (source, parameter details, purpose). However, it lacks details on return format or potential limits (e.g., rate limiting, pagination).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description's Args section documents both parameters: doi with an example and limit with default/max constraints. This compensates for the lack of schema detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves papers citing a given paper, using Semantic Scholar. It directly contrasts with sibling tools like search_papers (general search) and get_paper_details (single paper info).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing citation analysis but does not explicitly state when to avoid this tool or name alternatives. No when-not guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_author_profileA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the data source (OpenAlex) and the type of data returned, which adds some transparency. However, it lacks details about error handling, authentication needs, rate limits, or whether the operation is read-only. With no annotations, the description carries the full burden but provides only partial disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence stating the core function, one line about the data source, and an Args section with a brief parameter description. Every sentence adds value without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, no output schema), the description adequately covers the purpose and data types. It lacks explicit mention of output structure, but the examples of data (publication counts, citations, fields) provide sufficient context for an agent to understand the tool's function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'name' is explained with a description and a concrete example ('Yoshua Bengio'), adding meaning beyond the schema's property title. Although schema coverage is 0%, the description compensates effectively, though it could be more precise about name format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for an author and displays profile information, including specific data types (publication counts, citation metrics, research fields). It distinguishes itself from sibling tools like search_papers or get_citations by focusing on author profiles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not specify when to choose this over get_citations or search_papers, nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_detailsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses data sources but omits other behavioral traits like read-only nature, error handling, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences clearly state purpose and data sources, followed by parameter details. Efficient and front-loaded with essential info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Describes what data is combined and from which sources, sufficient for an agent to infer return structure. Lacks mention of output schema or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds significant value over schema by defining DOI, providing an example, and explaining its role. Compensates for 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves complete paper metadata via DOI and distinguishes it from siblings by specifying the data sources (Crossref and Semantic Scholar).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when a DOI is available and full metadata is needed, but does not explicitly state when not to use it or provide alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_topicsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool searches OpenAlex, filters papers since 2023, and sorts by citation count. This provides meaningful behavioral insight beyond the tool name, though it does not mention rate limits, caching, or result structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two short paragraphs and a bulleted Args list. Every sentence adds value, and the main purpose is front-loaded in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description provides sufficient context: source database, time filter, sorting criteria, and parameter details. It could be improved by briefly noting what the output contains (e.g., paper titles and citation counts).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning to the topic parameter by providing examples ('large language models', 'CRISPR') and explaining it as a subject area. For limit, it clarifies the default (10) and maximum (50), which goes beyond the schema's basic type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds the most-cited recent papers on a topic, specifying the source (OpenAlex), time range (since 2023), and sorting (by citations). This distinguishes it from sibling tools like search_papers, which likely perform broader searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Ideal um Trend-Themen zu erkunden' (ideal for exploring trending topics), providing clear usage context. However, it does not explicitly mention when not to use this tool or suggest alternatives like search_papers for non-trending topics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.