Server Quality Checklist

Tool Scores

• get_paper_detailC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a retrieval operation ('조회합니다'), implying read-only behavior, but doesn't cover critical aspects like authentication needs, rate limits, error handling (e.g., invalid paper IDs), response format, or whether it's a real-time query. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose. It's front-loaded with the core action and resource, with no wasted words. However, it could be slightly more structured by explicitly separating the tool's function from the types of details retrieved.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a retrieval tool with 2 required parameters), no annotations, and no output schema, the description is incomplete. It doesn't explain what the output looks like (e.g., JSON structure), error conditions, or behavioral constraints. For a tool that fetches detailed information, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('paper_id' and 'source') with descriptions and required status. The description doesn't add any parameter-specific details beyond what's in the schema, such as examples of paper IDs or how 'source' affects the data retrieved. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('조회합니다' - retrieves/looks up) and resource ('논문/문헌의 상세 정보' - detailed information of papers/documents), specifying what information is retrieved (abstract, keywords, citation info). It distinguishes from 'search' (which likely finds papers) and 'list_providers' (which likely lists sources), though the differentiation could be more explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., needing a paper ID), when not to use it (e.g., for searching papers), or how it relates to sibling tools like 'search' (which might return brief results) or 'list_providers' (which might list available sources).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool performs integrated search across multiple databases but lacks details on behavioral traits such as rate limits, authentication requirements, pagination behavior, or what happens with partial matches. The description is minimal and doesn't adequately disclose operational characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences that efficiently state the tool's function and examples of search criteria. It's front-loaded with the core purpose, though it could be slightly more structured by explicitly mentioning it's a read-only operation or default behaviors.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the search returns (e.g., result format, fields included), how results are sorted, or error handling. For a search tool with multiple parameters and no structured output, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by listing examples of search criteria (keyword, author, year) but doesn't provide additional semantic context beyond what's in the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '통합 검색합니다' (integrated search) across multiple academic databases for papers/documents. It specifies the resource (academic papers/documents) and scope (multiple databases), though it doesn't explicitly differentiate from sibling tools like 'get_paper_detail' or 'list_providers'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions what the tool does but doesn't indicate scenarios where 'search' is appropriate versus 'get_paper_detail' (likely for retrieving specific paper details) or 'list_providers' (likely for listing available databases).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_providersB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the basic action ('retrieves a list') without mentioning any behavioral traits such as whether it's read-only, if it requires authentication, rate limits, or what the output format might be. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence in Korean that directly states the tool's purpose without any fluff or redundancy. It is appropriately sized and front-loaded, making it easy to understand at a glance. Every word earns its place, contributing to the core meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for effective tool use. It doesn't explain what the returned list contains (e.g., provider names, IDs, metadata), how it's structured, or any prerequisites. For a tool that might return complex data (academic databases), more context is needed to understand the output and usage implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any. A baseline score of 4 is given as it handles the lack of parameters correctly without unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '사용 가능한 학술 DB 목록을 조회합니다' (Retrieves a list of available academic databases). It specifies the verb '조회합니다' (retrieves) and the resource '학술 DB 목록' (academic database list). However, it doesn't explicitly distinguish this tool from its siblings (get_paper_detail, search), which might have overlapping academic contexts, so it doesn't reach a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention its siblings (get_paper_detail, search) or specify contexts where listing providers is preferred over searching or getting details. This lack of comparative usage information limits its effectiveness in tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.