Server Quality Checklist

Tool Scores

• reindexB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not explain side effects like potential downtime or performance impact during rebuilding. With destructiveHint=false, it fails to clarify that clearing the existing index (if clear_existing=true) is a destructive action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It conveys the essential purpose without extraneous details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is adequate. However, it lacks situational context (e.g., when reindexing is needed) and does not mention prerequisites like having downloaded papers.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the 'clear_existing' parameter with a description, achieving 100% coverage. The tool description adds no additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'rebuild' and the specific resource 'local semantic index for downloaded papers', distinguishing it from sibling tools like 'download_paper' or 'semantic_search'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to reindex the index, such as after downloading new papers or when search results are stale. No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_paper_latex_sectionsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are sparse: readOnlyHint is false (implying possible side effects, but the tool appears read-only), and openWorldHint is true (suggesting external effects, but none are described). The description provides no insight into behavior beyond the basic function, such as whether it may access external resources, handle missing LaTeX, or have performance implications. It does not contradict annotations, but it does not add meaningful context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core purpose. It contains no fluff and is easy to scan. It earns its place by being clear and direct without extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 params, no output schema), the description is minimally acceptable. The schema covers parameters well, but the description lacks details on the output format (e.g., how headings are structured) and edge cases (e.g., papers without LaTeX). For a tool that returns a list of headings, an agent might need more context, but it is not severely deficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter descriptions in the schema cover all three parameters (paper_id, start, max_sections) with reasonable details, achieving 100% coverage. However, the description itself adds little beyond the schema; it doesn't explain how start and max_sections affect the outline or provide examples. The baseline of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a compact outline of headings from LaTeX source, which is specific and distinguishes it from sibling tools like get_paper_latex_section (which likely fetches a single section) and get_paper_latex (which likely returns the full source). The verb 'list' and resource 'headings' are precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for obtaining a structural overview of a paper's LaTeX source, but it does not explicitly state when to prefer this over get_paper_latex or get_paper_latex_section, nor does it mention when not to use it (e.g., for papers without LaTeX source). The sibling names provide some context, but the description itself lacks direct guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_latex_sectionA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=false and openWorldHint=true, but the description does not elaborate on behavioral traits like side effects, error handling, or what happens if the section is not found. It adds minimal context beyond what annotations already imply, so a score of 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence of 12 words with no filler. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with no output schema, the description is adequate but lacks information on return format (e.g., plain LaTeX text) and default max_chars behavior. Given the schema covers parameters, the description could be slightly more complete, but it meets the minimum.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so each parameter is already documented. The description adds context for section_id ('by outline ID or exact title') but does not significantly enhance understanding beyond the schema. Baseline 3 is correct.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return', the resource 'one bounded LaTeX section', and the method 'by outline ID or exact title'. It distinguishes this tool from siblings like list_paper_latex_sections (which lists sections) and read_paper (full paper).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving a specific section, but does not explicitly state when to use this tool versus alternatives, such as using list_paper_latex_sections to find the section ID first or read_paper for multiple sections. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_latexA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and openWorldHint=true. The description adds 'safely process, cache,' which discloses caching behavior and sanity processing, going beyond the annotation basics. This adds useful context about latent side effects and safety guarantees, consistent with the non-read-only flag.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately conveys the purpose, includes a usage pointer, and has zero waste. It is front-loaded with the core action and keeps white space minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only 3 parameters with full schema coverage and a simple return value (LaTeX source text). The description mentions the bounded nature and clearly states the return type. It also provides an alternative usage (section tools), making it adequately complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all parameters thoroughly. The description mentions 'bounded,' which aligns with max_chars but does not add significantly new meaning. Since the schema handles parameter semantics well, the description provides only marginal supplementary context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns bounded original LaTeX source, distinguishing it from sibling section tools. The verb 'Download, safely process, cache, and return' is specific about the action and resource. However, 'safely process' and 'cache' add some ambiguity about the core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Use section tools for targeted reading,' which provides a clear alternative. This implies this tool is for getting the full source (bounded) as opposed to section-level access. It lacks an explicit 'don't use when' clause, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_paperA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors beyond annotations: it tries HTML first then falls back to PDF conversion, and it stores the paper locally. Annotations provide `openWorldHint: true`, and the description adds specifics about the fallback strategy and local storage, which informs the agent of side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with front-loaded purpose. Each sentence adds essential information: core function, fallback mechanism, and pagination/storage details. There is no redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a tool with moderate complexity. However, it lacks details about the output format (text content is mentioned but no structure), error handling, prerequisites (e.g., arXiv API access), or rate limits. An output schema is missing, so the description could compensate by describing the return value more precisely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage, and the description adds context by explaining that `start` and `max_chars` are used for pagination of very large papers. This helps the agent understand when to use these optional parameters, going beyond the basic schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool downloads a paper from arXiv and returns text content. It specifies the resource (arXiv paper) and the action (download, return text). It also distinguishes from siblings like `get_abstract` or `read_paper` by mentioning HTML/PDF fallback and pagination for large papers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for full-text extraction and mentions pagination for large papers, but it does not explicitly guide on when to use this tool versus siblings like `get_paper_latex_section` or `get_abstract`. There is no 'when to use' or 'when not to use' language.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• citation_graphA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, so the bar is lower. The description adds value by specifying that it returns both citing and referenced papers, which is not in the annotations. It doesn't mention pagination or output format, but with annotations covering safety, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler. Every word contributes to conveying the tool's function, making it highly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, read-only, open-world), the description is sufficient. It could specify the return format (e.g., list of paper objects), but that is not critical given the context. A 4 reflects minor additional detail that could be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for paper_id, so the schema already documents the parameter. The tool description does not add any extra meaning beyond what the schema provides, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and a clear resource ('papers citing an arXiv paper and papers that it references'), explicitly distinguishing this tool from siblings like search_papers (search) or read_paper (content access). It unambiguously states the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use the tool: when citation or reference information is needed. It does not explicitly contrast with alternatives or say when not to use it, but the purpose is self-evident given the sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_abstractA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and openWorldHint=true. The description adds value by clarifying it saves tokens and does not download, reinforcing the read-only nature. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sentences, each with clear purpose: action, usage, returns, workflow. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch tool with one parameter, the description sufficiently covers return values and workflow context. Missing error handling or format details, but adequate for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for paper_id. The description does not add additional meaning beyond restating the parameter purpose. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (fetch), the resource (abstract and metadata of arXiv paper by ID), and distinguishes from sibling 'download_paper' by emphasizing it does NOT download the full paper.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using before download_paper to assess relevance and save tokens, and provides a workflow tip. Lacks explicit when-not-to-use for other siblings like read_paper or get_paper_latex_section, but the guidance is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_citationsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint, openWorldHint), the description adds significant behavioral context: uses authoritative metadata, never model-generated fields, preserves version suffixes, deterministic keys, and returns status/error per paper. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose. Every sentence provides value: first sentence defines the action and data source, second covers edge cases and limitations. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter input, no output schema, and annotations present, the description covers all essential aspects: purpose, data source, behavior (deterministic keys, version handling), return content, and format limitation. Complete for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'paper_ids' is fully described in the input schema (format, constraints). The description adds context about using authoritative metadata but does not add new parameter-level semantics beyond schema. Schema coverage is 100%, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it exports BibTeX citations using authoritative arXiv metadata. The verb 'export' and resource 'BibTeX citations' are specific, and the scope (arXiv papers, authoritative metadata) differentiates it from siblings like download_paper or get_abstract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'BibTeX only; RIS/CSL-JSON are not yet supported,' providing a key limitation. It implies when to use this tool (when BibTeX is needed) but does not explicitly state when not to use it or list alternative tools. The guidance is clear but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_paperA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only declare readOnlyHint=true. The description adds substantial behavioral context: returns markdown, supports pagination (start/max_chars), and fails with clear error if not downloaded. This goes well beyond the annotation and fully informs the agent of tool behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: purpose and output format, pagination and error handling, workflow. Front-loaded with the most critical information, no unnecessary words. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains the return format (markdown) and covers pagination and prerequisites. For a 3-parameter tool with 100% schema coverage, this description is fully complete—it informs the agent of everything needed to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all three parameters. The description mentions 'start/max_chars pagination' but adds no new parameter details beyond what the schema provides. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads text content of previously downloaded papers, distinguishing it from get_abstract (which returns only the abstract) and download_paper (which downloads but doesn't read). The verb 'read' and resource 'paper' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the prerequisite (call download_paper first) and provides a workflow (search_papers -> download_paper -> read_paper). It notes the error condition if not downloaded. However, it does not explicitly mention when to use an alternative like get_abstract for small summaries, which would make it a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_searchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds the important behavioral constraint that it only searches the local downloaded collection, which goes beyond the annotations' readOnlyHint. It also mentions the pro dependency requirement. However, it does not detail behavior for edge cases like providing both query and paper_id, or the output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, with two sentences that front-load the core functionality and then provide usage guidance. Every sentence adds necessary information, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's scope and prerequisites well. However, it lacks information about the output format (e.g., what fields are returned). Given that there is no output schema, the description could be more complete by mentioning the expected return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented in the schema. The description adds value by providing examples for query ('attention mechanisms for long sequences') and clarifying the purpose of paper_id (finding similar papers). This enriches the semantic understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs semantic similarity search over locally downloaded papers, distinguishing it from search_papers (which searches arXiv). It provides specific use cases (free-text queries or similar to paper_id), making the purpose very clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use this tool: only after downloading papers via download_paper. It warns about empty results if no papers are downloaded and directs the user to use search_papers first. It also mentions pro dependencies, providing comprehensive usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_alertsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses the timestamp update side effect, the incremental nature of results ('subsequent calls only return newer papers'), and the return structure. Annotations indicate readOnlyHint, and the description does not contradict this; it clarifies the side effect is metadata-only. This goes beyond surface-level description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sentences, each adds critical information. No redundancy, front-loaded with the main purpose, then covers parameters, side effects, prerequisites, and output. Efficient and structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with one optional parameter and no output schema, the description covers prerequisites (watch_topic), behavior (timestamp update), and return value (summary + metadata). The context is complete for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter, fully described in schema. The description adds value by explaining the exact matching requirement ('must match the topic string used in watch_topic exactly') and the behavioral difference between omitting vs. providing it. Coverage is 100%, so the description enriches but doesn't need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific language: 'Check all saved topic watches for newly published papers' with clear scope. It distinguishes itself from siblings by focusing on saved watches rather than general search or retrieval. The optional topic parameter is explained with behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear when-to-use guidance: 'Omit to run all watches, pass a topic to check only that watch.' Also explicitly directs users to call watch_topic first. It lacks an explicit alternative tool comparison but the contextually relevant usage is well covered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_papersA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds behavioral details beyond annotations: default sort is relevance, rate limiting is auto-handled, and on rate limit error wait 60 seconds. Aligns with readOnlyHint and openWorldHint. No contradiction, but lacks explicit statement about output format (no output schema exists).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is long but well-organized with clear sections (Query Construction, Advanced Patterns, Category Filtering, etc.). Each section adds value, though some examples could be trimmed without loss. Front-loads purpose effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 6 parameters and no output schema, the description is exceptionally complete: covers query optimization, category catalogs, date ranges, sort options, rate limits, and research tips. Provides all necessary context for effective usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters, but description massively enriches semantics: detailed query syntax with quoted phrases, OR, ANDNOT, field-specific searches, category code expansions with meanings, date format examples, and optimal parameter usage patterns. Far exceeds schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Search for papers on arXiv with advanced filtering and query optimization' – a specific verb+resource with explicit scope. It distinguishes from siblings like get_abstract and read_paper by emphasizing search and filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Extensive guidelines cover query construction, field-specific searches, category filtering, date filters, sort selection, and rate-limiting behavior. It gives explicit advice on when to use relevance vs. date, and when to use category filters, making tool selection and invocation clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watch_topicA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses behavioral traits beyond annotations: it is persistent, updates on duplicate topic strings, and returns only new papers since last check. Annotations indicate it's not read-only or destructive, which the description confirms and enriches.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with each sentence adding value. It front-loads the main purpose and follows with behavior, syntax, and usage advice.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is nearly complete but lacks details on the return value or confirmation of the save/update action. Since there is no output schema, a brief mention of what the tool returns would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good descriptions. The description adds significant meaning by stating the topic string uses search_papers syntax, providing examples, and explaining the update behavior on same topic.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool saves or updates a persistent research topic watch, distinguishing it from check_alerts which retrieves results. It uses specific verbs like 'save or update' and specifies the resource (persistent watch).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on pairing with check_alerts and explains the query syntax with examples. However, it does not explicitly state when not to use this tool, though the context and sibling tools make it obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_papersA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the edge case of an empty list when no papers are downloaded, and clarifies that only arXiv IDs are returned, not full text. With minimal annotations (readOnlyHint only), this adds valuable behavioral context without contradiction. It sets expectations for the response format and the need to call read_paper for content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, using three short, information-dense sentences plus a workflow arrow. Every sentence adds value with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-argument tool with no output schema, the description covers everything an agent needs: what is listed, the output format, edge case behavior, and the surrounding workflow context. No further details are required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema fully documents the interface. The baseline for no parameters is 4, and the description adds no parameter-specific details because none are needed. It doesn't detract, hence a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists downloaded papers and explicitly notes the return type (arXiv IDs only) and differentiates from related tools by directing to read_paper for content. The workflow arrow further situates the tool in the pipeline, leaving no ambiguity about its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a workflow (search_papers -> download_paper -> list_papers -> read_paper) and tells the user when to use an alternative ('use read_paper to access content'). This serves as both when-to-use and when-not-to-use guidance, going beyond a simple definition.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.