Server Quality Checklist

Tool Scores

• analyze_relevanceA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the input must be from search_papers and returns sorted results with a summary, but lacks details on side effects, error handling, or internal behavior (e.g., AI usage). The output schema exists but is not described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the purpose, followed by typical usage and parameter details. No extraneous information, every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two parameters and an output schema exists, the description adequately covers the workflow and parameter expectations. It does not detail return structure (handled by output schema) or error conditions, but is reasonably complete for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds significant meaning: topic should be 1-3 English sentences with an example, and papers_json must be the JSON from search_papers containing a papers array. This compensates well for the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes relevance of papers to a research topic and returns sorted results with a summary. It distinguishes itself from siblings like search_papers (retrieval) and get_paper_detail (individual paper) by specifying the workflow and output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit usage guideline: first call search_papers, then pass its JSON to this method. It clearly indicates the typical use case but does not explicitly mention when not to use it or alternative sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_detailA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It mentions automatic external source fetching, which is key. However, it does not discuss error handling, network failures, or rate limits, leaving some gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a brief statement of purpose followed by a structured argument list. Every sentence adds value, and no extra information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 optional parameters, output schema present), the description covers core functionality and parameter usage. It lacks details on error handling and edge cases, but overall is complete enough for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description compensates by explaining each parameter's purpose, format expectations, and constraints (e.g., 'title and URL are mutually exclusive', engine choices with default). This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets detailed info of a single paper and automatically fetches the full abstract from an external source. This distinguishes it from sibling tools like search_papers (which returns multiple results) and analyze_relevance (which performs analysis).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides guidance on when to use each parameter (title exact match better, URL restricted to specific engine, engine options with default and fallback). It implies usage for single-paper details but does not explicitly contrast with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_papersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses rate limiting and retry behavior. Does not mention auth requirements, side effects, or behavior when no results found. Basic transparency is present but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise, with purpose stated first, followed by constraints and parameter details. No redundancy. Could be slightly shorter, but effective structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all parameters, rate limiting, and engine behavior. Output schema exists, so return value explanation is unnecessary. Does not mention input validation or edge cases, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description explains all 5 parameters in detail: query (with example), num_results (range and default), year_low/high (meaning), engine (options). Adds significant value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states '搜索学术论文' (search academic papers) with specific verb and resource. It distinguishes from sibling tools such as analyze_relevance, get_paper_detail, etc., which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance on engine choices: bing (no proxy), google (needs proxy), auto (failover). Also mentions built-in rate limiting. However, does not explicitly state when to use this tool versus siblings, though siblings serve different functions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_relevance_chartA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It discloses that a local HTTP server is started, the link is returned, and the message field contains markdown and raw URL. It does not mention potential side effects or error handling, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with clear sections: main purpose, what charts are generated, server details, return format, and usage. It is slightly verbose but front-loaded with the key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 2 parameters and output schema (not shown but present), the description covers input format, output behavior, and prerequisite call to analyze_relevance. Missing details about error cases or server availability, but sufficient for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions (0% coverage). The description fully compensates by explaining that 'topic' is the research topic description (chart title) and 'papers_json' must be the JSON string from analyze_relevance containing a 'ranked_papers' array.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates multi-angle relevance analysis charts, launches a local HTTP server, and returns a link. It details three specific chart types (relevance bar chart, K-Means clustering scatter plot, TF-IDF keyword analysis), which distinguishes it from siblings like analyze_relevance or search_papers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to first call analyze_relevance and then pass its JSON output to this tool. It provides a typical usage pattern but does not explicitly state when not to use it or mention alternative tools for other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.