Server Quality Checklist

Tool Scores

• paper_ai_analyzeA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully discloses critical behavioral traits: external API dependency (via env vars), the conditional PDF download attempt with abstract fallback, and JSON output format. However, it omits other behavioral details like timeout behavior, API cost implications, or caching policies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (functionality, environment setup, fallback logic, Args/Returns). The Args/Returns sections are necessary given the schema's lack of descriptions, though they echo structural fields. Every sentence conveys essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (covering return structure) and only one input parameter, the description is sufficiently complete for its complexity level. It covers the essential external dependency setup and conditional behavior (PDF fallback) needed for an agent to understand operational requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by documenting the single parameter: it explains 'doi' represents the paper's DOI and provides a concrete example ('10.1109/tim.2021.3106677'), which aids the LLM in generating correct invocations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool uses AI to analyze papers and specific outputs expected (core contributions, methods, findings). It implicitly distinguishes from siblings like `paper_download` (which merely fetches files) and `paper_search` (which finds papers) by emphasizing AI-driven content analysis, though it could explicitly contrast with these alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implicit guidance by noting it requires OpenAI-compatible API environment variables and explaining the fallback behavior (PDF full-text vs abstract). However, it lacks explicit guidance on when to choose this over `paper_download` or whether users should prefer manual download for large batches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_batch_downloadA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully discloses the return format (JSON string with status and statistics) and file output behavior (PDF), but omits critical safety information like whether existing files are overwritten or error handling behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses a clear docstring structure (Args/Returns) that frontloads the purpose statement and organizes supplementary information efficiently. The Chinese text is appropriately concise with zero redundant sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter tool with simple primitives, the description adequately covers inputs and acknowledges the output schema (JSON result). It appropriately delegates detailed return structure documentation to the output schema itself.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description compensates effectively: it explains 'dois' is a list with concrete examples, and clarifies 'output_dir' is a directory path with default value (current directory). It could improve by detailing DOI format validation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action verb (批量下载/batch download) and clear resource (多篇论文 PDF/multiple paper PDFs), immediately distinguishing it from the singular 'paper_download' sibling tool through the emphasis on batch processing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the Args section implies usage through the plural 'dois' parameter and_examples, there is no explicit guidance on when to choose this over 'paper_download' (singular) or prerequisites like valid DOI format requirements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_healthA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It specifies the return value format ('JSON string of health status') and the external dependencies checked, but omits behavioral traits like whether results are cached, if the check makes live HTTP requests, rate limiting, or safety properties (read-only nature).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two distinct sections: the purpose statement naming specific data sources, and a clear 'Returns' declaration. Every sentence earns its place with zero redundancy or tautology.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter simplicity and existence of an output schema (per context signals), the description is reasonably complete: it identifies the scope (which specific sources are checked) and return format. Minor gaps remain regarding error handling semantics and cache duration, but sufficient for a utility health endpoint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100% (trivially). Per the rubric, zero parameters warrants a baseline score of 4. The description correctly focuses on behavior rather than inventing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('检查/check') with specific resources ('论文下载服务各数据源的可用性/availability of paper download service data sources'), explicitly listing the monitored sources (Unpaywall, arXiv, Sci-Hub mirrors). This clearly distinguishes it from sibling tools like paper_download or paper_search which perform operations on papers rather than checking service health.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the purpose is clear, there is no explicit guidance on when to use this tool versus alternatives (e.g., 'call before batch downloads to verify availability'). The usage is implied by the health-check nature, but lacks explicit when/when-not recommendations or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  披露了两源合并去重的行为特性，并详细说明了返回JSON的结构（包含title, authors, journal等字段）。由于无注解，描述承担了完整的行为说明责任，但未提及错误处理、认证需求或速率限制。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  采用标准docstring结构，分行为说明（双源搜索）、参数(Args)和返回值(Returns)三部分，中文表述简洁，每句都提供有价值信息，无冗余。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  鉴于无注解且Schema描述覆盖率0%，描述已相当完整：涵盖两个参数语义、双源去重行为、返回JSON结构。但因未提及错误处理情形、速率限制或与兄弟工具的明确区分，略有不足。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  输入Schema描述覆盖率为0%，但描述通过Args部分完美补偿：详细说明query参数接受关键词或DOI并提供具体示例（'gradient magnetic field coil'或'10.1109/tim...'），rows参数说明为返回结果数量且默认值为8。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述明确说明'搜索论文'并使用Crossref + Semantic Scholar双源合并去重，提供了具体动词和资源。但并未明确区分与paper_recommend等兄弟工具的差异。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  说明了支持关键词搜索或DOI查询两种模式（支持...或...），但未提供何时使用此工具而非paper_recommend或paper_citation_graph的指导，也未说明使用限制或前置条件。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_citation_graphA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses data source (Semantic Scholar API), recursive fetching behavior (depth levels explained), output format (Mermaid + JSON), and rate-limiting hints via max_per_level. Missing explicit error behavior or auth requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Clear Args/Returns structure with logical flow. Chinese text is appropriately concise; no redundant sentences. Minor deduction for Returns section listing output fields that may duplicate output schema details, though helpful given context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex graph-generation tool with zero schema coverage, description adequately covers inputs, data source, and output structure (nodes/edges/statistics). Sufficient since output schema exists to handle return value specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by documenting all 3 parameters: doi with example syntax, depth with semantic meanings (1=direct, 2=secondary), and max_per_level with clear quantity semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb '生成' (generates) + resource '论文引用图谱' (paper citation graph) and explicitly distinguishes from siblings via the Mermaid visualization and Semantic Scholar API sourcing—unique traits not shared with paper_search or paper_download.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage through functional description (use when needing citation visualization), but lacks explicit 'when to use vs alternatives' guidance comparing to siblings like paper_ai_analyze or paper_search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_downloadA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries significant burden by disclosing the source priority chain (Unpaywall → arXiv → Sci-Hub) and return JSON structure (success, path, size_mb, source). However, it omits rate limiting behavior, network failure handling, and explicit confirmation that files are written to disk (though implied by '保存 PDF').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The Args/Returns structure is logical and scannable. The essential information (action + source chain) appears in the first sentence. Minor deduction for including return field documentation when an output schema exists, though this redundancy aids readability given the tool's critical file-I/O nature.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, file I/O) and absence of annotations, the description adequately covers inputs, outputs, and sourcing behavior. The presence of an output schema reduces the burden to describe return values, yet it provides this detail anyway.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage (schema only contains titles), the description fully compensates by providing semantic meaning for both parameters: doi includes a concrete example ('10.1109/tim.2021.3106677'), and output_dir explains the purpose ('保存 PDF 的目录路径') and default behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the specific action (download PDF via DOI), resource type (academic paper), and unique mechanism (multi-source fallback chain: Unpaywall → arXiv → Sci-Hub). This specificity naturally distinguishes it from siblings like paper_batch_download (single vs. multiple) and paper_search (retrieval vs. discovery).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description implies single-paper usage through the singular '一篇论文' (a paper) and specific DOI parameter, it lacks explicit guidance on when to prefer paper_batch_download for multiple papers or prerequisites like needing a valid DOI upfront. Usage is inferred but not expressly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paper_recommendA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral disclosure burden. It successfully explains the internal pipeline (file scanning → feature extraction → keyword mapping → search) and output structure (JSON with detected libraries/terms, queries, paper list). It lacks explicit read-only confirmation or side-effect warnings, though 'analyze' and 'scan' imply non-destructive behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Excellent structure with three distinct sections: main purpose (2 sentences), Args documentation, and Returns documentation. No redundancy; every sentence contributes to understanding the tool's function, parameters, or output format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-stage feature extraction across file types) and 0% schema coverage, the description provides sufficient context: it explains the extraction targets (import libraries, algorithm terms, LaTeX titles), documents both parameters, and summarizes the output JSON structure without needing to repeat the full output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates. The Args section documents workspace_path with a concrete example ('E:/半导体实验') and top_n with its default value (8), providing complete semantic meaning for both parameters that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool '分析工作区代码，自动推荐相关学术论文' (analyzes workspace code, automatically recommends academic papers) with specific mechanism details (scanning .py/.tex/.md, extracting imports/terms). It distinguishes from siblings like paper_search (manual query) and paper_download (specific paper retrieval) by emphasizing the code-driven feature extraction workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The implicit usage context is clear from the detailed mechanism description—use this when you have a codebase to analyze for paper recommendations. However, it lacks explicit when-not-to-use guidance or direct comparison statements like 'use paper_search instead for manual keyword queries'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.