Server Quality Checklist

Tool Scores

• search_arxivC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states the basic action ('搜索' - search) without mentioning any behavioral traits like whether it returns metadata only, supports pagination, has rate limits, requires authentication, or what format results come in. This leaves significant gaps for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise ('搜索 arXiv 论文' - Search arXiv papers) - just four characters in Chinese. While this is efficient and front-loaded, it might be overly terse given the tool's functionality and lack of sibling differentiation. Every word earns its place, but more content could be justified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and the description is minimal, it's incomplete for a search tool with 2 parameters. The description doesn't explain what kind of search it performs, what fields are searched, what the results look like, or any limitations. With richer context needed, this falls short of being complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('query' and 'maxResults') with descriptions. The tool description adds no additional meaning about parameters beyond what's in the schema. According to guidelines, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '搜索 arXiv 论文' (Search arXiv papers) states the basic purpose with a clear verb ('搜索' - search) and resource ('arXiv 论文' - arXiv papers). However, it lacks specificity about what kind of search it performs (e.g., title/abstract/full-text) and doesn't distinguish it from sibling tools like 'get_recent_ai_papers' or 'parse_paper_content'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this search tool is appropriate compared to 'get_recent_ai_papers' (which might fetch recent papers without a query) or 'parse_paper_content' (which processes paper content). No context about use cases or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_arxiv_pdf_urlC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't reveal any behavioral traits like whether it's read-only, requires authentication, has rate limits, or what the output format might be. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese ('获取 arXiv PDF 下载链接') that directly states the tool's purpose with zero waste. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a PDF URL string), potential errors, or behavioral context, leaving gaps that could hinder an AI agent's ability to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the input parameter clearly documented as accepting arXiv URLs or IDs. The description doesn't add any meaning beyond what the schema provides, such as examples or constraints, so it meets the baseline score of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '获取 arXiv PDF 下载链接' clearly states the verb ('获取' meaning 'get') and resource ('arXiv PDF 下载链接' meaning 'arXiv PDF download link'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_recent_ai_papers' or 'parse_paper_content' which also work with arXiv papers, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_arxiv' or 'parse_paper_content'. It doesn't mention prerequisites, such as needing an arXiv URL or ID, or exclusions, leaving the agent to infer usage from the input schema alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_paper_contentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the HTML/PDF fallback behavior, which is useful. However, it doesn't describe critical traits like whether this is a read-only operation, potential rate limits, authentication needs, error handling, or what the parsed output looks like (since there's no output schema). For a tool that likely involves network requests and content processing, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence) and front-loaded with the core purpose. Every word earns its place by specifying the action, resource, and a key behavioral detail (HTML/PDF preference). There's no wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (parsing academic papers with fallback logic), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what 'parsed content' includes (e.g., full text, abstracts, sections), error conditions, or performance characteristics. For a tool with 2 parameters (one nested) and no structured output definition, more context is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('input' as arXiv URL/ID and 'paperInfo' as optional metadata). The description adds no additional parameter semantics beyond what's in the schema. It doesn't explain format expectations (e.g., arXiv ID patterns) or how 'paperInfo' might affect parsing. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('解析论文内容' - parse paper content) and specifies the resource (papers from arXiv via URL/ID). It distinguishes from siblings like 'get_arxiv_pdf_url' (which only retrieves URLs) and 'search_arxiv' (which searches metadata) by focusing on content extraction. However, it doesn't explicitly mention what content is extracted (e.g., text, sections, references).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context by stating '优先使用 HTML 版本，回退到 PDF' (prefer HTML version, fallback to PDF), which implies when to use this tool for content parsing rather than just getting PDF URLs. However, it doesn't explicitly compare with alternatives like 'get_recent_ai_papers' or 'search_arxiv', nor does it specify prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_ai_papersB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool fetches recent papers but doesn't describe key behaviors such as rate limits, authentication needs, return format (e.g., list of papers with metadata), pagination, or error handling. This leaves significant gaps for an AI agent to understand how to invoke it correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese that directly states the tool's purpose without any wasted words. It is front-loaded with the core action and resource, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimal but adequate for basic understanding. However, it lacks details on behavioral aspects like return format, error handling, or usage constraints, which are important for complete contextual understanding. With no output schema, the description should ideally hint at what is returned, but it doesn't, leaving gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there are no parameters to document. The description adds value by specifying the arXiv category (cs.AI/recent), which isn't in the schema. This earns a baseline score of 4 for parameter semantics, as it compensates appropriately for the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('获取' meaning 'get') and resource ('arXiv AI 领域最新论文' meaning 'latest arXiv AI papers'), specifying the exact arXiv category (cs.AI/recent). It distinguishes from sibling tools like 'search_arxiv' by focusing on recent papers rather than search functionality. However, it doesn't explicitly contrast with 'get_arxiv_pdf_url' or 'parse_paper_content', keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving recent AI papers from arXiv, but provides no explicit guidance on when to use this tool versus alternatives like 'search_arxiv' for broader searches or 'get_arxiv_pdf_url' for specific PDFs. The context is clear but lacks explicit when/when-not statements or named alternatives, resulting in an implied usage score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.