Server Quality Checklist

Tool Scores

• reindexB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not explain side effects like potential downtime or performance impact during rebuilding. With destructiveHint=false, it fails to clarify that clearing the existing index (if clear_existing=true) is a destructive action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It conveys the essential purpose without extraneous details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is adequate. However, it lacks situational context (e.g., when reindexing is needed) and does not mention prerequisites like having downloaded papers.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the 'clear_existing' parameter with a description, achieving 100% coverage. The tool description adds no additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'rebuild' and the specific resource 'local semantic index for downloaded papers', distinguishing it from sibling tools like 'download_paper' or 'semantic_search'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to reindex the index, such as after downloading new papers or when search results are stale. No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_paper_latex_sectionsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are sparse: readOnlyHint is false (implying possible side effects, but the tool appears read-only), and openWorldHint is true (suggesting external effects, but none are described). The description provides no insight into behavior beyond the basic function, such as whether it may access external resources, handle missing LaTeX, or have performance implications. It does not contradict annotations, but it does not add meaningful context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core purpose. It contains no fluff and is easy to scan. It earns its place by being clear and direct without extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 params, no output schema), the description is minimally acceptable. The schema covers parameters well, but the description lacks details on the output format (e.g., how headings are structured) and edge cases (e.g., papers without LaTeX). For a tool that returns a list of headings, an agent might need more context, but it is not severely deficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter descriptions in the schema cover all three parameters (paper_id, start, max_sections) with reasonable details, achieving 100% coverage. However, the description itself adds little beyond the schema; it doesn't explain how start and max_sections affect the outline or provide examples. The baseline of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a compact outline of headings from LaTeX source, which is specific and distinguishes it from sibling tools like get_paper_latex_section (which likely fetches a single section) and get_paper_latex (which likely returns the full source). The verb 'list' and resource 'headings' are precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for obtaining a structural overview of a paper's LaTeX source, but it does not explicitly state when to prefer this over get_paper_latex or get_paper_latex_section, nor does it mention when not to use it (e.g., for papers without LaTeX source). The sibling names provide some context, but the description itself lacks direct guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_paper_sectionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already covers the safety profile, so the description only needs to add meaningful context. It does add clarity about section boundaries and markdown output, but it does not describe behavior like truncation, the effect of return_full_text, or what happens when the section is not found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler. The key idea is front-loaded, and the scoping detail about sibling and parent sections adds useful information without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with fully documented parameters, the description provides enough context: a single markdown section, selected by outline ID or title, without extra surrounding sections. An explicit pointer to sibling alternatives would improve completeness, but the description is not inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already explains all parameters. The description repeats the meaning of section_id as outline ID or unique title, but it provides no additional semantic value beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action (return), the resource (one bounded markdown section), and the selector (outline ID or unique title). It also differentiates from full-paper or outline-related siblings by stating that sibling and parent sections are not included.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving a single section rather than an entire paper, but it does not explicitly state when to choose it over alternatives like read_paper, get_paper_outline, or get_paper_latex_section. The boundary statement is useful but not a full usage guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_latexA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and openWorldHint=true. The description adds 'safely process, cache,' which discloses caching behavior and sanity processing, going beyond the annotation basics. This adds useful context about latent side effects and safety guarantees, consistent with the non-read-only flag.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately conveys the purpose, includes a usage pointer, and has zero waste. It is front-loaded with the core action and keeps white space minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only 3 parameters with full schema coverage and a simple return value (LaTeX source text). The description mentions the bounded nature and clearly states the return type. It also provides an alternative usage (section tools), making it adequately complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all parameters thoroughly. The description mentions 'bounded,' which aligns with max_chars but does not add significantly new meaning. Since the schema handles parameter semantics well, the description provides only marginal supplementary context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns bounded original LaTeX source, distinguishing it from sibling section tools. The verb 'Download, safely process, cache, and return' is specific about the action and resource. However, 'safely process' and 'cache' add some ambiguity about the core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Use section tools for targeted reading,' which provides a clear alternative. This implies this tool is for getting the full source (bounded) as opposed to section-level access. It lacks an explicit 'don't use when' clause, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_latex_sectionA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds useful behavior beyond annotations: 'bounded' output, whitespace/case-normalized matching, and macro-expanded sections. The readOnlyHint false annotation is not contradicted; the wording 'Return' implies a read operation, though side effects are not explicitly ruled out.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence that front-loads the core behavior and then packs the key matching details into one parenthetical. There is no filler or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is nearly sufficient for a simple retrieval tool with fully documented parameters, but there is no output schema and the return payload is not described beyond 'section'. It also lacks guidance on parent flags such as start and max_chars beyond what the schema already provides.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are fully documented in the schema, so the baseline is 3. The description enriches section_id semantics by showing that section lookup supports either an ID or a title and that matching is normalized while macros are expanded.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'return one bounded LaTeX section'. It also names the two lookup keys (outline ID or title), which distinguishes this tool from siblings like get_paper_latex, list_paper_latex_sections, and get_paper_outline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose makes it clear that this is for retrieving a single LaTeX section, but it does not explicitly say when to prefer it over alternatives. It also omits that section_id should usually come from list_paper_latex_sections, which the schema notes but the description could reinforce.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• citation_graphA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and openWorldHint annotations, the description reveals concrete behavior: results are bounded by default to stay within the unauthenticated quota, an API key can raise the limit, and persistent rate limits return status=rate_limited instead of failing hard. These are valuable, non-obvious behavioral details from the agent's perspective.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states exactly what the tool returns, and the second covers the important operational constraints. There is no filler or repetition of structured metadata.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description provides enough about the return set ('papers citing' and 'papers referenced') and the special status behavior. It could also mention how max_citations applies to each citation/reference grouping, but the core context needed to select and invoke the tool is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully documents both parameters, including the default of 50 for max_citations. The description does not add significant new parameter-level semantics beyond reminding the caller about quota/rate-limit context, so the baseline for high schema coverage is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb and resource: 'return papers citing an arXiv paper and papers that it references' using Semantic Scholar's citation graph. It names both the input kind (arXiv paper) and the operation, making the tool separable from sibling tools like list_papers or search_papers at a glance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for citation and reference graph traversal, which is a clear use case, but it does not explicitly contrast it with sibling tools like export_citations or search_papers. The operational context about rate limits is useful, but the 'when to use this instead of that' guidance is only implied, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_paper_textA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given annotations only provide readOnlyHint=true, the description adds valuable non-obvious behaviors: suppression of high-overlap near-duplicates, preference for section-diverse hits, and the use of bounded substring matching. It does not fully describe the output format or error conditions, but what it provides goes clearly beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly written in three sentences with no wasted repetition. The core purpose is front-loaded, then the deduplication/diversity behaviors are added, and finally the technology-focused scoping is stated. Each sentence contributes useful selection and invocation information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only search tool, the description covers the main behaviors: scope to downloaded papers, substitution search, bounded results, section offsets, and near-duplicate suppression. It does not explicitly describe failure behavior for missing downloads or give a detailed return shape, but with no output schema and a simple matcher, the provided context is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers all 4 parameters at 100%, including case-insensitive substring behavior and default values for max_passages and passage_chars. The description adds context about bounds and diversity but does not need to restate parameter details; the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action — searching a downloaded paper for bounded matching passages with section/source offsets — which clearly defines what the tool does. It also differentiates via 'substring search' and 'no Torch' from semantic_search and search_papers siblings, making the tool's specific role apparent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: it is for substring search within a single downloaded paper, implying use when the agent needs exact lightweight matching rather than semantic similarity or cross-paper search. However, it does not explicitly name alternatives or state when not to use the tool, so it stops slightly short of full usage routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unwatch_topicA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false, so the mutation is known. The description adds value beyond that by disclosing the exact-match requirement and the explicit not-found error behavior, which are behavioral details not present in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, each carrying unique value: the main action, the matching constraint, and the error/fallback behavior. Ends with a pointed pointer to list_watches, with zero filler words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-param delete tool, the description covers the core behavior, the error path, and the recommended pre-use step. The lack of an output schema is partially offset by the not-found error description; the only minor gap is the success return shape, which is not critical given the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents the topic parameter well. The description reinforces the exact-match concept and the relationship to watch_topic, but adds no new parameter syntax, formatting, or constraint beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States the verb 'Delete' with a precise resource ('saved topic watch') and the exact matching mechanism. This clearly distinguishes it from siblings like list_watches (inspect) and watch_topic (create), so an agent knows which tool to pick.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives concrete guidance: 'Use list_watches to inspect saved watches before deleting,' which is a practical pre-step and names the relevant sibling. It stresses the exact-match prerequisite but doesn't explicitly cover when-not-to-use scenarios relative to other siblings like check_alerts, so it stops at clear context with no exclusion list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_outlineA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint=true annotation, the description adds 'paginated', 'Stable hierarchical section IDs', and the markdown context. These are meaningful behavioral details. It does not disclose failure modes, but the read-only nature is already annotated and there is no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with zero filler. The core behavior is front-loaded, and the second sentence adds the most actionable routing hint an agent needs. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a fully documented input schema, a read-only annotation, and a clear description of the returned outline and its IDs, the tool is fully specified for correct invocation. The pointer to read_paper_section completes the usage contract without needing an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so every parameter is already well documented. The description adds a light link to pagination semantics but no deeper parameter detail, which is acceptable given the high schema coverage baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb ('Return'), a precise resource ('heading outline for a downloaded paper (markdown)'), and distinguishes itself from section content retrieval by explicitly pointing to read_paper_section. It is immediately clear what the tool produces and how it differs from its siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage clue by saying 'use read_paper_section to fetch one', signaling that this tool is for outlines, not content. It does not exhaustively enumerate all alternatives, but the routing guidance is sufficient for a heading-list tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_citationsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint, openWorldHint), the description adds significant behavioral context: uses authoritative metadata, never model-generated fields, preserves version suffixes, deterministic keys, and returns status/error per paper. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose. Every sentence provides value: first sentence defines the action and data source, second covers edge cases and limitations. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter input, no output schema, and annotations present, the description covers all essential aspects: purpose, data source, behavior (deterministic keys, version handling), return content, and format limitation. Complete for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'paper_ids' is fully described in the input schema (format, constraints). The description adds context about using authoritative metadata but does not add new parameter-level semantics beyond schema. Schema coverage is 100%, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it exports BibTeX citations using authoritative arXiv metadata. The verb 'export' and resource 'BibTeX citations' are specific, and the scope (arXiv papers, authoritative metadata) differentiates it from siblings like download_paper or get_abstract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'BibTeX only; RIS/CSL-JSON are not yet supported,' providing a key limitation. It implies when to use this tool (when BibTeX is needed) but does not explicitly state when not to use it or list alternative tools. The guidance is clear but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_papersA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description explicitly says this does not do a live re-fetch, reads from local metadata only, returns an empty list when nothing has been downloaded, and describes the compact mode. These are meaningful behavioral disclosures that help the agent predict behavior without invoking the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured, and front-loaded with the core action. Every sentence adds value: what is listed, source of data, return fields, compact behavior, empty-list behavior, and workflow placement. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter set, readOnlyHint annotation, and absence of an output schema, the description covers what an agent needs to know: what is returned, the data source, the compact option, the empty-list edge case, and the intended workflow. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (the compact parameter is fully documented in the schema). The description restates the default and compact behavior but does not add new meaning beyond the schema. Baseline 3 is appropriate because the schema already carries the parameter semantic burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource—'List all papers that have been downloaded and stored locally via download_paper'—and explicitly distinguishes it from live re-fetching. It also names the exact fields returned, making the tool's purpose unmistakable and distinct from sibling search/read tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear workflow context: 'search_papers -> download_paper -> list_papers -> read_paper', which tells an agent when in the workflow this tool applies. It does not explicitly name when not to use it or point to alternatives like search_papers for live results, but the 'no live re-fetch' phrasing and workflow chain effectively convey the intended usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_alertsA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description extensively discloses nontrivial behavioral details: the drain cursor advances, has_more/check_start/last_checked semantics, truncation behavior, cursor reset conditions, and not-found errors. These go far beyond the annotations, which only indicate non-read-only and non-idempotent behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although the description is long, every sentence contributes operational detail: purpose, parameter behavior, cursor mechanics, prerequisites, error cases, and return summary. It is front-loaded with the core purpose and the rest earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single optional parameter, no output schema, and stateful behavior, the description is remarkably complete. It explains how the tool mutates internal state across calls, what the client can expect in terms of pagination, error semantics, and the response summary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though the schema already describes the topic parameter with 100% coverage, the description adds crucial semantic nuance: omission runs all watches, a topic string must match exactly, and calling with a nonexistent topic yields a clear error. This meaningfully extends the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Check all saved topic watches for newly published papers since the last check.' It clearly distinguishes itself from sibling tools like watch_topic and search_papers by centering on alerting for saved watched topics, not general search or watch creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear operational context: when to use it (checking saved watches for new papers) and a prerequisite ('Use watch_topic to register topics before calling'). It lacks explicit comparisons to alternatives, but the intended usage is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_paperA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations only say readOnlyHint=false and openWorldHint=true, so the description carries the burden for behavior. It discloses meaningful side effects: it stores locally, permits overwriting cached versions, and can return truncated paginated responses. It also explains fallback behavior between HTML and PDF conversion, which annotations cannot capture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loads the core action before diving into edge cases. Every sentence earns its place; the pagination, cache overwrite, and full-text options are all tightly packed without rambling.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately covers what an agent needs for correct invocation: return length cap, truncation behavior, continuation semantics, and cache invalidation. The operational contract is complete enough to use the tool without additional research.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all parameters with 100% coverage, so the baseline is 3. The description adds value beyond schema by explaining the bounded default (~12,000 chars), the interaction between start and next_retrieval, and the real-world use at force=true. This raises it slightly above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('download'), a precise resource ('a paper from arXiv'), and the delivered output ('text content'). It clearly explains retrieval strategy (HTML first, PDF fallback) and local storage, making the tool's purpose distinguishable from simple reading siblings like read_paper.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear operational guidance: how to handle truncated results with start and return_full_text, and when force=true is needed to overwrite a cached paper. It does not explicitly route away from siblings like read_paper or search_paper_text, so it lacks explicit exclusions, but the usage context is otherwise clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watch_topicA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses important behavioral traits beyond annotations: new watches seed last_checked to creation time, first checks do not return historical matches, and calling with same topic updates the existing watch. This meaningfully explains the watch lifecycle without contradicting the provided annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is thorough but not bloated. It starts with the core purpose, then explains the behavioral subtleties, then gives query syntax and examples, and ends with the companion tool. Each sentence serves a clear decision-making purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description covers nearly everything needed to invoke the tool correctly: state persistence, update semantics, initial last_checked seeding, query syntax, and check_alerts integration. The only small gap is what the save/update call itself returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all three parameters well, and the description adds value by treating the topic string as the identity key for updates and giving concrete query examples. It could mention categories and max_results in the body, but the schema covers those adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Save or update a persistent research topic watch.' It clearly differentiates watch_topic from search_papers, check_alerts, list_watches, and unwatch_topic by defining its persistent, update-in-place behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says when to use this tool and explicitly pairs it with check_alerts for polling, and clarifies that reusing the same topic string updates the watch instead of creating duplicate. However, it does not explicitly state when to prefer search_papers for one-time queries rather than setting up a persistent watch.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_abstractA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint and openWorldHint annotations already cover the safety profile. The description adds extra behavioral context by explicitly stating it fetches metadata without downloading the paper, enumerating the returned fields, and advising usage as a relevance-screening step. There is no contradiction with the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each with a distinct purpose: what it does, when to use it, and what it returns. There is no filler; it is front-loaded and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple metadata-fetching tool with one parameter, readOnly annotations, and no output schema, the description covers invocation context, return fields, and conditional usage. Nothing important is missing for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter paper_id is fully described in the schema with a type and example. The description merely mentions 'by arXiv ID,' which adds no meaning beyond the schema. With 100% schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb and resource: 'Fetch abstract and metadata by arXiv ID,' and immediately differentiates itself by noting 'without downloading the paper,' which distinguishes it from the sibling download_paper. Listing the returned fields (title, authors, abstract, categories, published date, PDF URL) makes the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: 'Use before download_paper to assess relevance' and 'After compact search, use for one full abstract; skip if search used abstract_mode=full.' This provides both when-to-use and when-to-skip guidance, clearly distinguishing it from alternate flows.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_watchesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate this is read-only, and the description reinforces that by noting it does not update last_checked and does not check for new papers. This adds meaningful behavioral clarity beyond what the annotation alone provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, leading with the core purpose and then adding behavioral notes and sibling references. Every sentence earns its place with no redundant detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool, the description fully covers what is returned, what it does not do, and how it relates to sibling tools. Nothing essential is missing for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter behavior to describe. The description appropriately focuses on the semantics of the operation itself, which is sufficient given the empty input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all saved topic watches and explicitly notes it does not check for new papers, which distinguishes it from check_alerts. It also enumerates the returned fields, making its purpose precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: use this to inspect saved watches, not to poll for updates. It also names sibling tools for related actions (unwatch_topic and check_alerts), which helps route the agent correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_papersA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool readOnly and open-world, but the description adds substantial behavioral detail: arXiv's ~3s rate-limit policy, 60s retry on rate-limit, default/cap values, response metadata fields, and the interaction between abstract_mode and follow-up calls. This is the transparency an agent needs beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well organized, front-loading the main purpose and then grouping related instructions: query construction, date/sort/pagination, search mode rules, and rate limits. Every sentence carries operational information; nothing is filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is thorough for such a complex tool: it covers query syntax, defaults, response metadata fields, pagination, and rate limiting. It falls just short of explicitly describing the shape of each result item (e.g., paper ID, title, authors), which would fully close the gap created by the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, yet the description goes beyond field listings by explaining query syntax semantics, default behaviors, pagination protocol, and projections. Details like 'Unprefixed terms match title+abstract (not authors)' and 'Pass next_start with the same abstract_mode' genuinely add operating knowledge that the schema alone does not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb and resource: 'Search arXiv by query', and immediately lists the optional dimensions (categories, date range, sort, pagination). It clearly distinguishes itself from sibling tools like list_papers and semantic_search by targeting arXiv keyword queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong contextual guidance: how to structure queries, which prefixes to use, which categories are valid, and how pagination should be chained via next_start. It also tells the agent when to use get_abstract instead of requesting full abstracts, though it does not explicitly enumerate sibling exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_paperA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral detail beyond the readOnlyHint annotation: chunking at ~12,000 characters, continuation via is_truncated and next_start, the return_full_text opt-out, and the failure condition when the paper has not been downloaded. It also covers the typical workflow context, making the behavior predictable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and every sentence contributes useful guidance: what the tool reads, how chunking works, how to continue or override truncation, the failure precondition, and the intended workflow. The critical behavior is front-loaded and there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description covers the important response behaviors (Markdown, truncation, continuation token) and the prerequisite download step. For a paginated reader tool with no output schema, this provides enough contextual information for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is already 100%, but the description adds meaning beyond the schema by explaining the default 12,000 character bound, the fact that start should come from a prior truncated response, and that return_full_text=true retrieves the entire remaining paper. This is exactly the kind of cross-parameter context an agent needs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: read the text content of a previously downloaded paper and return it in markdown format. It also distinguishes this tool from download_paper by making the download prerequisite explicit, and the scope is specific enough to separate it from section or abstract readers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly tells the agent when the tool can be used: only after download_paper has been called, and it even provides the workflow search_papers -> download_paper -> read_paper. It does not explicitly contrast this tool with sibling alternatives such as read_paper_section or get_paper_outline, which is the only gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already establish read-only and local-only behavior, and the description adds important operational context beyond them: empty results when no papers are downloaded, plus the required pro dependency installation command. This is valuable behavioral information for an agent deciding whether the tool is available and what preconditions apply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient and front-loaded. It leads with the core scope, then the important failure condition, then the prerequisite workflow, and finally the installation dependency. Each sentence adds genuine information without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only search tool with no required parameters and only three well-documented schema properties, this description covers the critical information: preconditions, empty-result behavior, supported query modes, and the needed setup command. No output schema is present, but return-value shape is not a blocker for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes all three parameters with 100% coverage, so the description is not required to explain them. It adds value by giving a concrete free-text example and clarifying that query and paper_id are the two supported semantic-search modes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states a specific operation — semantic similarity search — and scopes it to papers already downloaded locally via download_paper. It identifies two query modes (free-text or paper_id) and is immediately distinguishable from sibling tools like search_papers and search_paper_text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains when the tool returns empty results, recommends search_papers to discover arXiv papers, then download_paper to add them to the local index before semantic_search. This gives the agent a clear workflow and names the alternative tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.