Server Quality Checklist

Tool Scores

• nl_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true. The description adds that the tool performs an actual network round trip and checks for the auth key, which provides useful behavioral context beyond what annotations offer. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise Korean sentence that front-loads the tool's purpose ('연결 점검') followed by specific details. There is no waste or unnecessary repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description sufficiently covers its purpose and behavioral aspects. It does not specify the return format, but for a status check this is likely inferred and not critical. Adequate for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is complete and coverage is 100%. The description correctly does not include parameter details, and the baseline of 4 applies for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the tool as a connection check ('연결 점검') and specifies that it verifies authentication key presence and performs one actual round trip to the search API. This clearly sets it apart from sibling tools nl_search and nl_collect, which are focused on searching and collecting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies that this tool is for checking connectivity and auth status before using search/collect tools. While it does not explicitly state 'use this before nl_search', the context is evident from the tool name and description, providing clear usage context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nl_searchA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and openWorldHint annotations, the description discloses critical behaviors: silent fallback for unsupported search targets, token matching instead of partial title match, severe recall loss with exact=True, pagination cap at 500, and the meaning of total/truncated/records. This far exceeds annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but densely packed with essential warnings and empirical data. It is well-organized with bullet points and section breaks, making it navigable. However, it could be slightly trimmed without losing critical information, as some examples are verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description clarifies the meaning of return fields (total, truncated, records, cap_hit) and warns against misinterpreting empty records. It covers all parameters and edge cases for a complex search tool. Missing details like per-record field names are not critical for selecting/invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description is the sole source of parameter meaning. It explains kwd, exact, srch_target (listing supported values and fallback behavior), category (including error case), rows (with limits), and extra_params. It also covers pagination behavior, fully compensating for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with '[소장자료 검색] 국립중앙도서관 소장자료를 검색한다', clearly stating the tool searches the National Library of Korea's collection with a specific verb and resource. However, it does not explicitly distinguish from sibling tools nl_status and nl_collect, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage guidance: exact=True is only for finding a specific known material, not for corpus collection ('자료를 넓게 모을 때는 쓰지 말고, 전체 표제를 아는 특정 자료 조회에만 쓸 것'). It also warns against unsupported srch_target values and the '전체' category, effectively stating when not to use certain options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nl_collectA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations, the description discloses many behavioral traits: local post-processing for year filters ('year_from/year_to/contains 는 로컬 후처리다'), the 500-record cap, inability to achieve full enumeration ('전수는 여전히 불가능하다'), and side effects like file saving and multiple API calls. It also details measured failure rates for `exact`, providing valuable real-world context. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a one-line summary followed by parameter-by-parameter explanations and clearly marked warnings (⚠️). Although long, every sentence provides essential information or empirical data, and the use of bold and bullet-like formatting improves readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (16 parameters, no output schema), the description covers almost every aspect, including meta output fields (`meta.cap_hit_terms`, `meta.year_missing_dropped`, `meta.axes[].partition.unreachable`), failure modes, and performance metrics. The only omissions are `name` and `srch_target`, but these are relatively minor and inferred by name.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates thoroughly, explaining nearly all 16 parameters including `terms`, `kwd`, `exact`, `category`, `auto_partition`, `partition_depth`, `contains`, `formats`, `save`, `out_dir`, `year_from/year_to`, `extra_params`, and `max_records`. It adds critical meanings such as the union logic and local filtering behavior that schema alone cannot convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with '[수집] 검색어들을 각각 조회해 **합집합**으로 모으고 파일로 저장한다' (collects queries into a union and saves to file), clearly stating the tool's function. It distinguishes from sibling tools by emphasizing saving to file and union aggregation, which is not mentioned in nl_search or nl_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides extensive usage guidance: warnings against using `exact` for corpus collection ('코퍼스 수집에는 쓰지 말 것'), instructions for `auto_partition` to bypass the 500 limit, and trade-offs for `partition_depth`. However, it does not explicitly compare with sibling tools `nl_search` or `nl_status` to clarify when to choose this tool over them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.