Server Quality Checklist

Tool Scores

• download_paperC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states the basic action of downloading a PDF, without explaining side effects (e.g., file creation), output format, or whether any authentication is needed. This is insufficient for a tool that likely creates files on the system.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no redundant wording, which is structurally neat. However, it is under-specified; the brevity comes at the cost of explanatory value, so it does not fully earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three parameters (two required), no parameter descriptions, and an output schema that is not detailed, the description should compensate by explaining usage and context. It only states the action, omitting crucial information such as how to choose a platform, what to pass for id, and what the output schema represents. The presence of siblings makes the lack of guidance more problematic.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the three parameters (id, dir, platform). It is unclear what 'id' refers to (paper ID?), what 'dir' does (destination directory?), and what values 'platform' can take. This leaves the agent reliant on guessing parameter meanings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '从指定平台下载论文PDF文件' clearly states the tool downloads a paper PDF from a specified platform, using a specific verb (download) and resource (paper PDF). It distinguishes from siblings like search_papers and read_paper by focusing on the download action, though it lacks detail on what exactly is returned or saved.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as search_papers or read_paper. It does not mention prerequisites, when a download is appropriate, or any exclusions, leaving the agent to infer usage solely from the tool name and sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_paperB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that PDF text content is read, but does not mention potential errors, rate limits, authentication, or what the output actually contains. The single sentence lacks sufficient behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy, front-loading the core purpose immediately. Every word contributes to the meaning, making it highly concise and well-structured for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters, no annotations, and no parameter descriptions. The description is too minimal to provide complete context: it doesn't explain the role of id or dir, nor does it describe return values or expected output format, even though an output schema exists. The context is incomplete for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. However, it only hints at 'platform' being specified, while 'id' and especially 'dir' remain unexplained. The description adds little meaning beyond what the schema property names imply, leaving the dir parameter particularly ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb (读取/read) and resource (PDF text of papers), and mentions 'specified platform', which directly maps to the platform parameter. This distinguishes it from sibling tools like search_papers, download_paper, and recommend_platforms, which have different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention any prerequisites or context. It simply states what the tool does, leaving the agent to infer usage from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_papersB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that the tool searches papers on a specified platform and does not mention any behavioral traits such as pagination, rate limits, authentication requirements, or result format. This minimal disclosure is insufficient for a tool with potential side effects or complex behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, efficiently front-loaded with the primary action and immediately providing a relevant pointer to recommend_platforms. Every word earns its place; there is no fluff or repetition of the title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, nested filters object, and an output schema), the description is severely incomplete. It lacks any mention of return values, parameters, pagination, or usage scenarios beyond the platform recommendation. The agent cannot confidently invoke this tool based solely on this description, especially with no annotation support.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 6 parameters with 0% description coverage, so the description must compensate by explaining parameter meaning. It only references the platform implicitly ('specified platform') and does not describe query, year, limit, cursor, or filters. This leaves the agent without enough semantic understanding of the parameters, warranting a low score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool searches for academic papers on a specified platform, using the specific verb 'search' and resource 'academic papers'. It distinguishes from siblings like recommend_platforms, download_paper, and read_paper by focusing on the search action, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs using recommend_platforms to get recommended platforms, providing a clear prerequisite for when to use this tool. However, it does not explicitly mention when not to use this tool or how it relates to download_paper/read_paper, leaving some gaps but still offering actionable context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_platformsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  无注解，描述承担行为披露责任。它披露了返回形式（按优先级排序的平台列表及其特点），但未提及可能的失败情况、输出格式限制或额外行为。基本透明但不深入。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  单句描述，信息完整，无冗余内容，且结果输出形式在开头前置。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  该工具功能相对简单且存在输出 schema，描述提及输入依据和返回列表形式，足以提供基本使用概念。但缺少参数细节、使用边界和替代工具对比，整体完整度中等。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  schema 描述覆盖率为 0%，描述需要补偿参数语义。'查询意图和学科领域'大致对应 query 和 field，但未说明具体参数名、field 枚举值如何影响推荐结果，也没有说明 query 的用法，语义贡献有限。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述以明确的动词 '推荐' 指向特定资源 '学术搜索平台'，并说明基于查询意图和学科领域，与兄弟工具 search_papers（搜索论文）、download_paper（下载）、read_paper（阅读）清晰区分。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述隐含了使用场景（需要推荐平台时），但没有明确说明何时应使用本工具而不是 search_papers 等替代工具，也没有给出排除条件。从名称和兄弟工具可推断，但描述本身未提供直接指导。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.