Server Quality Checklist

Tool Scores

• analyze_trendsC
  2.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description only mentions analyzing trends without detailing how it works, what data it accesses, potential limitations, or output format. It lacks critical behavioral traits like data sources, processing methods, or any constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that efficiently states the tool's function. It's front-loaded with the core purpose, though it could be more structured if it included additional details. There's no wasted text, making it appropriately sized for its limited content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a trend analysis tool with 2 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'analyze' involves, what the output looks like, or how parameters affect the analysis. This leaves significant gaps for an AI agent to understand and use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'specific category' which relates to the 'category' parameter, but doesn't explain the 'days' parameter or provide any additional meaning beyond the schema's basic titles. This leaves half the parameters inadequately described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the purpose ('analyze trends') and specifies a resource ('specific category'), but it's vague about what 'analyze' entails and doesn't distinguish from siblings like 'search_papers' or 'scrape_recent_category_papers'. It provides a basic function but lacks specificity about the analysis scope or output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search_papers' or 'scrape_recent_category_papers'. The description implies usage for trend analysis in a category, but it doesn't specify prerequisites, exclusions, or comparative contexts with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_infoC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states it 'gets detailed information' but doesn't describe what that includes (e.g., title, authors, abstract), whether it's a read-only operation, potential errors (e.g., invalid ID), or performance aspects. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's appropriately sized for a simple tool and front-loaded with the core purpose. Every part of the sentence contributes directly to explaining the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a read operation with one parameter), lack of annotations, and no output schema, the description is incomplete. It doesn't specify what 'detailed information' includes, error handling, or return format, making it inadequate for an agent to use the tool effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It mentions '논문 ID' (paper ID) as the parameter, which adds some meaning beyond the schema's generic 'paper_id' property. However, it doesn't explain the format, constraints, or examples (e.g., numeric, alphanumeric, DOI), leaving the parameter poorly documented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the purpose: '논문 ID로 상세 정보를 가져옵니다' (Get detailed information by paper ID). It specifies a verb ('가져옵니다' - get/fetch) and resource ('상세 정보' - detailed information), but is vague about what 'detailed information' entails. It doesn't differentiate from sibling tools like 'search_papers' or 'scrape_recent_category_papers'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a paper ID), exclusions, or comparisons to sibling tools like 'search_papers' (which might retrieve multiple papers) or 'scrape_recent_category_papers' (which might fetch recent papers by category).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_papersC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions searching arXiv papers by keyword but lacks behavioral details such as rate limits, authentication needs, result format, pagination, or error handling. This is inadequate for a search tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Korean that directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (search operation with 2 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects, parameter details, or return values, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only mentions '키워드' (keyword) implicitly, without explaining the 'max_results' parameter or providing any additional context like format constraints or usage examples. This fails to address the undocumented parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('검색합니다' - searches) and resource ('arXiv 논문' - arXiv papers) with a specific method ('키워드로' - by keyword). It doesn't distinguish from sibling tools like 'scrape_recent_category_papers' or 'get_paper_info', but the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'scrape_recent_category_papers' or 'get_paper_info'. The description implies usage for keyword-based searches but doesn't specify contexts, exclusions, or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_recent_category_papersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions scraping, implying network operations and potential rate limits or permissions, but doesn't detail these aspects, response format, or error handling. The description adds minimal context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action ('scrape') and purpose. It avoids unnecessary details, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a scraping tool with no annotations, no output schema, and low parameter coverage, the description is incomplete. It lacks details on behavior, output format, error cases, and integration with siblings, making it inadequate for full agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It implies 'category' is used to specify the target and 'max_results' limits output, but doesn't explain valid categories, format, or constraints. This adds some meaning but doesn't fully bridge the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it scrapes a 'recent' page for a specific category to fetch the latest paper list. It uses specific verbs ('scrape', 'fetch') and identifies the resource ('recent page', 'paper list'), but it doesn't explicitly differentiate from sibling tools like 'search_papers' or 'get_paper_info', which might handle similar data differently.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions scraping a 'recent' page, but doesn't specify contexts, exclusions, or compare to siblings like 'search_papers' or 'analyze_trends'. This leaves the agent without clear usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.