Server Quality Checklist

Tool Scores

• semantic_scholar_get_authorB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint. The description adds that publications are optional, which is useful, but it doesn't detail related behaviors such as default limits or response structure. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It states the core action and a key option efficiently, earning its place without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, and the description omits important context such as the required author_id, how to control the optional publications (include_papers, papers_limit), and the default response format. For a tool with multiple parameters and no output schema, this is insufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% at the top level, and the description never mentions any parameters. While the nested schema has some parameter descriptions, the tool description itself does not compensate for the low coverage, leaving the agent to infer parameter meaning solely from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' with a clear resource 'author profile' and notes an optional publications list. This distinguishes it from sibling search tools, though it doesn't explicitly name alternatives. The purpose is clear and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like semantic_scholar_search_authors or semantic_scholar_get_paper. It doesn't mention that an author_id is required or that this is for retrieving a specific existing author rather than searching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_author_batchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and openWorldHint, so the safety profile is covered. The description adds the 1000-item limit, which is useful, but it does not disclose response format, error behavior, or partial-failure semantics. It provides modest value beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the verb and includes the key constraint. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a batch retrieval tool with no output schema, the description is minimal but adequate for basic understanding. It does not explain return value structure or error handling, but the max-1000 limit and read-only nature are clear. Given the simplicity of the operation, this is acceptable but not rich enough for a higher score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description does not explicitly document the parameters. The schema itself contains detailed descriptions for author_ids and response_format, but the description merely implies the list of IDs with 'multiple authors' and 'max 1000'. It does not compensate for the low coverage and adds no real semantic value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve multiple authors') and adds the key constraint 'single request (max 1000)', which distinguishes it from the sibling tool semantic_scholar_get_author. The verb and resource are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is only implied by the name and description—it is a batch variant of get_author, but no explicit guidance is given about when to prefer it over repeated single calls or when not to use it. No alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_bulk_papersA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description adds the max 500 limitation but does not disclose potential partial failures, missing IDs, rate limits, or response format. This is acceptable but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately communicates the core purpose and constraint. There is no wasted wording, and it is well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, and annotations cover safety, but the description is sparse. With no output schema, the agent lacks information about return values, error behavior, or what happens if any paper ID is invalid. The max 500 is helpful, but more context could be provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not describe any parameter semantics; it only mentions 'multiple papers' which vaguely relates to the paper_ids list. The context signal indicates schema description coverage of 0%, so the description must compensate, but it does not. The schema's internal definitions are not enough per the provided context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Retrieve multiple papers in a single request (max 500).' The verb 'Retrieve' with resource 'multiple papers' is specific, and the max limit distinguishes it from singular fetch (semantic_scholar_get_paper) and search (semantic_scholar_search_papers).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching multiple papers at once, especially with the 'single request' phrase, but it does not explicitly name alternatives or state when not to use it. It offers only implicit guidance regarding batch retrieval, not clear exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_snippet_searchA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint. The description adds useful behavioral context beyond annotations with the explicit rate-limit warning ('heavily rate-limited without an API key') and clarifies the return format ('text snippets with context'). This is meaningful additional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: two purposeful sentences plus a note. It front-loads the core action and return type, then a key caveat. Every word earns its place; there is no redundancy or padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters and no output schema, the description is under-specified. It does not explain result formatting, pagination, how to scope searches (e.g., by paper IDs or fields), or any usage examples. The rate-limit note is helpful but leaves too many operational gaps for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is reported as 0% at the top level, yet the description provides no parameter-level guidance. It only indirectly relates to the api_key parameter via the rate-limit note. The description fails to compensate for the lack of schema parameter descriptions, leaving filters like year, field_of_study, and paper_ids unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search') with a clear resource ('paper full text') and return type ('text snippets with context'). The title 'Search Paper Full Text' aligns, and it distinguishes from siblings like semantic_scholar_search_papers by focusing on full-text snippet retrieval rather than metadata search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context that this tool is for searching within full text and returns snippets, implying when to use it over sibling metadata search tools. However, it does not explicitly name alternatives or state when not to use it, so it lacks the exclusionary guidance seen in higher-scoring examples.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_bulk_searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint=true, covering safety and side-effect transparency. The description adds useful behavioral context by explaining that this tool supports cursor-based pagination and returns a continuation token, and that it is intended for large result sets. This goes beyond the annotations without contradicting them, providing the agent with a clearer operational picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, composed of two short sentences. It front-loads the primary purpose ('Search papers') and immediately conveys the key differentiators. Every sentence earns its place, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with many parameters and no output schema, the description provides the essential differentiating context (sorting, pagination, large result sets) and relies on the schema for parameter details. It is sufficient for the agent to select and invoke the tool correctly, though it does not describe the response format or pagination workflow in detail. Given the annotations and schema richness, the description is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description only alludes to sorting and pagination conceptually, but does not explain any of the actual parameters (e.g., year, fields_of_study, limit, token, response_format). With schema_description_coverage at 0%, the description fails to compensate for the low coverage. Although the schema itself contains parameter descriptions, the tool description offers little additional semantic meaning beyond the name of the feature.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search papers with sorting and cursor-based pagination for large result sets.' It uses a specific verb (search) and a resource (papers), and distinguishes this from regular search by highlighting sorting and pagination capabilities. The differentiation is clear even without naming the sibling tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts this tool with 'regular search' by stating 'Unlike regular search, supports sorting... and returns a continuation token for paging through all results.' This implies when to use this tool (when sorting or full pagination is needed) and when to use regular search (simple queries). However, it does not explicitly name the alternative sibling tool, so there is a slight gap in giving a fully explicit alternative name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_match_paperA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, openWorldHint, and idempotentHint, covering safety and side-effect expectations. The description adds behavioral context by stating it returns a match score, which is not in the annotations. This is helpful for the agent to know what output to expect. It does not discuss edge cases like no match or ambiguous matches, but the strong annotation coverage reduces the need for further disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary intent and return value. Every word is purposeful, with no redundancy or filler. It is an excellent example of concise, structured description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (one required parameter, no output schema, strong annotations), the description covers the essential purpose and return type. It does not explicitly state how to handle edge cases like no match or ambiguous titles, but these are likely predictable for a title-matching tool. The presence of response_format and api_key parameters is handled by the schema, so the description need not repeat them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool description provides no parameter information, and schema_description_coverage is 0%. Although the input schema includes rich descriptions (e.g., query is 'Paper title to match'), the rubric requires the description to compensate for low coverage. Since it does not mention any parameters or how to fill them, the description adds no value beyond what the schema already offers. This is a clear gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Find the single best paper matching a title string') and clearly distinguishes itself from sibling tools: it returns a single best match rather than a list (unlike search_papers) and operates on a title string rather than an ID (unlike get_paper). This is concise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when you have a title string and want the single best matching paper. It does not explicitly name alternatives, but the phrase 'single best paper' contrasts with search tools that return lists, providing clear context. No exclusions are mentioned, but given the straightforward use case, this is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover the safety profile with readOnlyHint, openWorldHint, and idempotentHint, which lower the burden. The description adds value by specifying the exact components checked (server health, API connectivity, key status), providing more detail than a generic 'check status.' It does not disclose return format or potential error conditions, but the annotations handle the critical safety aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that immediately conveys the tool's purpose without any unnecessary words or repetition. It is front-loaded and every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description covers the core functionality well by listing three specific status areas. However, it does not describe the response structure, which could be useful for an agent deciding how to interpret the result. Still, for a low-complexity tool with strong annotations, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is trivially complete. With no parameters to describe, the baseline for this dimension is 4, and the description does not need to add parameter-specific information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Check server health, API connectivity, and key status.' This clearly distinguishes it from the sibling tools, which are all focused on searching or retrieving papers/authors. There is no ambiguity about the tool's role as a status/health check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives, but the sibling context makes it obvious that this is the only status-related tool. The usage is implied rather than directly stated, and there is no guidance on prerequisites or scenarios where this should be called (e.g., before other API calls).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_paper_authorsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint. The description adds valuable behavioral context beyond those annotations: author ordering, limit bounds, default values, and Markdown-vs-JSON output options. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact—three sentences—and front-loaded with the core purpose. Every sentence earns its place by adding either differentiation, parameter detail, or output behavior, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with one required parameter and no output schema, the description adequately covers return contents, output format, author ordering, limit behavior, and ID formats. Minor omissions like error handling or rate limits are acceptable given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although context reports low schema description coverage, the description adds meaningful parameter context: a concrete paper_id format example, the 1-1000 limit with default 100, and response_format options. The schema also provides detailed per-parameter descriptions, so the two complement each other well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb and resource: 'Get full author profiles for a paper's authors.' It also explicitly distinguishes itself from the abbreviated author list in semantic_scholar_get_paper and connects to semantic_scholar_get_author via returned author IDs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear contrast with semantic_scholar_get_paper, implying when to use this tool over that one, and includes a concrete example paper_id. However, it does not explicitly enumerate exclusion cases or alternative tools for related scenarios, so it stops short of full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_export_citationA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, openWorldHint), the description discloses that the output is plain text with no response_format option, and lists specific error triggers: unknown ID, paper without citation data, or non-bibtex format. This is valuable behavioral context not covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise, front-loaded sentences: purpose, usage, behavior. Each sentence earns its place with no redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple citation-export tool with one required parameter, the description covers workflow (use after search/match), input requirements, output format, and error scenarios. With annotations covering safety and idempotency, no further context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides detailed descriptions for all parameters, including accepted paper_id formats and the format restriction. The description adds a concrete example ID and reinforces the error on non-bibtex, but doesn't substantially expand parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Export a citation for a paper in BibTeX format.' This clearly distinguishes it from sibling tools like search/get paper. It also states the output format and prerequisite.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use it after obtaining a paper ID from semantic_scholar_search_papers or semantic_scholar_match_paper, giving clear upstream context. However, it does not mention exclusions or alternative downstream tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_multi_recommendA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as readOnly, openWorld, and idempotent. The description adds valuable behavioral context: results are steered toward positives and away from negatives, malformed IDs raise an error before making requests, and the response format can be markdown or JSON. It stops short of discussing auth or rate limits, but the additional context is substantive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly written in four sentences, immediately stating the purpose, followed by usage guidance, an example, parameter ranges, and output format. There is no fluff—every sentence contributes essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a multi-seed recommendation tool, the description covers the central behavior, parameter constraints, error handling, and return format. It omits the deprecated api_key parameter, but the schema explains it. The tool has no output schema, so the description effectively fills the gap by stating what is returned (papers, markdown or JSON).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all fields, so the baseline is 3. The description enhances parameter understanding by showing an example with specific ID formats, reaffirming the 1-100 positive and 1-100 negative limits, and noting the limit/default/response_format options. This adds practical value beyond the schema's static descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get recommendations using multiple positive and negative example papers.' It immediately distinguishes itself from sibling tool semantic_scholar_recommendations by stating it is for 'more than one example,' making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly directs when to use this tool: 'Use instead of semantic_scholar_recommendations when steering with more than one example.' It also provides a concrete example and clarifies input limits, giving clear guidance on when and how to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_search_authorsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only/idempotent behavior, so the description adds value by disclosing Markdown default output, response_format='json' option, and pagination limits (max 100, default 10). It also warns about name ambiguity, offering extra operational context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: it opens with the core purpose, then supplies an example, caveats, alternatives, and paging/output details. Every sentence carries load, with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers primary usage, pagination, output formats, and points to relevant sibling tools. It does not detail the response structure, but given the schema and annotations, it is sufficient for selecting and invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already contains descriptive text for each parameter, so the description's example and mention of offset/limit and response_format add only modest value. The concrete query example and clarification of default response format help, but the schema does most of the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Search for academic authors by name' with a concrete example ('Yoshua Bengio'). It distinguishes from sibling tools by explicitly naming semantic_scholar_paper_authors for a different use case (listing authors of a specific paper).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly says when to use this tool (search by name) and when to use alternatives: semantic_scholar_get_author for confirming identity and semantic_scholar_paper_authors for listing paper authors. It also provides pagination guidance and response format selection, fully covering usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_search_papersA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint, covering safety and side-effect expectations. The description builds on this by adding search-specific behavioral details: boolean operators, quoted phrases, filter types, pagination via offset/limit, and default vs. JSON response format. It does not cover potential rate limits or error behaviors, but these are not critical for a read-only search tool when annotations already signal safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph that front-loads the core purpose, then packs capabilities, limitations, alternatives, and output format into just a few sentences. Every clause earns its place; there is no fluff or repetition of schema field names. The structure flows logically from what → how → when-to-use-other-tools → output format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple filters, pagination, output formats) and the absence of an output schema, the description fully equips an agent to select and invoke it correctly. It covers input essentials, pagination behavior, alternative tools for edge cases, and return format. The schema and annotations cover parameter details and safety, so no critical gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already contains detailed descriptions for every parameter (e.g., year formats, limit bounds, query examples). The tool description adds meaningful context that is not fully in the schema: relevance ranking semantics, the availability of specific filter families, and that response_format='json' yields raw JSON. Although the description does not enumerate each parameter by name, the schema provides that, and the description enhances understanding of when and how to use them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource ('Search for academic papers') and immediately clarifies the scope: relevance-ranked keyword search over 200M+ papers. It distinguishes itself from sibling tools by explicitly naming alternatives (bulk_search, snippet_search, match_paper) and explaining when each is appropriate, leaving no ambiguity about its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance by stating that for sorted or very large result sets one should use semantic_scholar_bulk_search, for full-text search one should use semantic_scholar_snippet_search, and for resolving a single known title one should use semantic_scholar_match_paper. It also notes pagination limits (max 100 per call), which is key for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_get_paperA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, openWorldHint, idempotentHint), the description discloses caching for 5 minutes, rate limits (1 req/s unkeyed, 10 with API key), automatic retry on 429/502/503, not-found errors, parallel fetching of citations/references, and default/alternate output formats. This is rich behavioral context that significantly helps the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized logically: what it does, accepted IDs, return fields, optional flags, caching/rate limits/errors, output format, and alternative tool. Each sentence adds distinct value without redundancy, making it informative yet efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description still lists the return fields, explains error behavior, caching, rate limits, and output format options. It also covers the main parameters and gives an explicit pointer to the bulk alternative, making the tool's behavior comprehensive for a single-paper fetch operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It explains accepted paper_id formats with an example, the meaning of include_citations/include_references and their 1-100 limit, and the response_format options. However, it does not explicitly mention api_key or the default/named limits for citations_limit/references_limit, so the coverage is good but not exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get paper details' and lists what it returns (title, abstract, authors, venue, year, citation counts, TLDR, PDF link), distinguishing it from siblings like semantic_scholar_search_papers and semantic_scholar_get_author. It also names the specific ID formats accepted, making the tool's scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'To fetch many papers at once use semantic_scholar_bulk_papers', naming the alternative for the bulk case. It also implies when to use this tool (when you need one paper's details, citations, or references) and explains key usage options like include_citations and response_format.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_scholar_recommendationsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint, openWorldHint, and idempotentHint, but the description adds substantial behavioral context beyond those: unknown seed IDs raise a not-found error, unkeyed requests are throttled to 1 req/s, 429/502/503 responses retry automatically with backoff, and the default output is Markdown. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and then efficiently covers usage, alternatives, errors, rate limits, and output format in a compact paragraph. Every sentence adds operational value, with no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and a nested parameter object, the description is remarkably complete. It explains the input requirements, the meaning of the main parameters, the sibling tool to use for richer steering, error behavior, throttling/retry, and how to switch between Markdown and raw JSON output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite the context showing 0% schema description coverage for the single top-level param, the description compensates thoroughly. It explains paper_id with a concrete example, clarifies limit as an upper bound, defines from_pool options ('recent' vs 'all-cs'), and explains response_format values. Only the deprecated api_key is not mentioned, but that is adequately covered in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get paper recommendations based on a seed paper.' It clearly distinguishes itself from the sibling tool by explicitly directing multi-example steering to semantic_scholar_multi_recommend, and it explains the candidate pool options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives concrete usage guidance: provide a known paper_id, use from_pool to choose the candidate set, and set limit. It explicitly names the alternative tool for multi-example scenarios. It also covers error handling and response format selection, making when-to-use very clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.