Server Quality Checklist

Tool Scores

• get_kolas_statisticsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the data source and that the tool returns statistics. However, it does not mention response format, potential errors, or rate limits. For a simple read-only tool, the behavioral disclosure is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences with no wasted words. It starts with the core purpose and immediately adds the data source and requirement, making it efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description could be more complete. It states the output is counts per category per year but omits details like response structure, pagination, or external API dependencies. It is adequate but leaves gaps for an agent to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only loosely connects 'per category per year' to the 'year' and 'category' parameters, but does not explain their formats, defaults, or the meaning of category values. The description adds minimal value beyond the schema's enum and pattern.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns annual KOLAS accreditation statistics (number of accredited organizations per category per year) and identifies the data source. It distinguishes itself from sibling tools ('get_lab_details' and 'search_accredited_labs') by being aggregate statistics vs. details or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions a required API key ('Requires KOLAS_SERVICE_KEY') as a prerequisite. It implies use for obtaining aggregate statistics, but does not explicitly state when to use this tool over siblings or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lab_detailsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses return data (metadata, scope table, history) and data source. However, it omits behavioral traits like idempotency, error handling, or potential side effects, making it adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first captures purpose and key qualifier, the second enumerates return content and data source. No redundant information; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one required parameter, no output schema, and no annotations, the description covers the purpose, parameter, return data, and data source. It lacks error states or rate limits, but for a simple lookup tool, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It mentions 'by accreditation number' which maps to the parameter, but does not explain the pattern or provide examples. This meets the minimum requirement but adds limited value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and resource ('full detail for a single KOLAS-accredited lab') along with the retrieval key ('by accreditation number'). It clearly distinguishes from siblings (statistics, search) by focusing on a single lab's comprehensive details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: use this to get detailed info for a specific lab, while siblings handle statistics or searching. However, it does not explicitly state when not to use or provide alternatives, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_accredited_labsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the data source (knab.go.kr) and that no API key is needed. However, it does not mention rate limits, authentication, or error handling. For a search tool, these are less critical, but the description is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence clearly states the action and criteria; the second adds return fields and data source. Front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description lists returned fields (accreditation number, name, category, scope fields, validity end date). It also notes the data source and no API key. Missing details like sorting or pagination behavior, but overall complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explains 'query' as free-text and 'category' as accreditation category, which adds meaning beyond the enum. It implies pagination via 'paginated list' but does not explain the 'page' and 'page_size' parameters or defaults. This partial compensation yields a score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search'), identifies the resource ('KOLAS-accredited laboratories'), and specifies the input methods (free-text query and accreditation category). It clearly distinguishes from sibling tools like 'get_kolas_statistics' and 'get_lab_details' by focusing on searching vs. statistics or details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching by query and category, and mentions a paginated list. While it doesn't explicitly state when not to use it, the context of sibling tools suggests appropriate usage. It could be improved by including exclusions, but it is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.