Server Quality Checklist

Tool Scores

• boj_get_data_codeA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses limits (250 series, 60k points) and error-handling behavior (auto-correction, suggestions). With no annotations, this is adequate but does not mention authentication, rate limits, or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a purpose: purpose, constraint, error handling. Concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, so description should explain return values; it hints at pagination via startPosition but lacks details on response format. For 6-param tool, sufficient but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with descriptions. Description adds cross-parameter constraints (same frequency) and system limits (max 250 codes), going beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches time series data by SERIES_CODE, which distinguishes it from siblings like boj_get_metadata (metadata) and boj_list_databases (listing). However, it does not explicitly contrast with boj_get_data_layer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage after obtaining metadata codes, and mentions constraints (same frequency, limits). But lacks explicit when-to-use or when-not-to-use guidance compared to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boj_get_data_layerA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses limits (1,250 series / 60,000 data points), wildcard support, and pagination via startPosition. It does not explicitly state read-only behavior, but the 'fetch' verb and lack of mutation language suffice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with front-loaded purpose: first sentence states action, second adds usage context, third gives key limits and features. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters and no output schema, the description covers main behaviors (limits, wildcard, pagination) but omits explanation of NEXTPOSITION origin and date format normalization details. Still fairly complete for a data retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about hierarchical browsing and limits but does not provide parameter-specific meaning beyond what the schema already includes, such as the layer format and wildcard use already detailed in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch' and resource 'time series data by layer', and explains the hierarchical structure and browsing by category, distinguishing it from sibling tools like boj_list_databases and boj_get_data_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: 'useful for browsing data by category within a database'. This guides the agent toward the intended use case, though it does not explicitly exclude other scenarios or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boj_list_databasesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses that 'No API call needed — uses a static list of 40+ databases,' which informs the agent about the tool's lightweight nature and scope. This adds value beyond expectations, though more details like update frequency could improve it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two sentences, front-loading the main purpose and usage instruction without extraneous detail. Every sentence is functional and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no output schema, optional parameters, static data), the description fully covers its purpose, usage context, and behavioral traits. No additional information is necessary for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters ('search' and 'category') having descriptions. The tool description adds no extra parameter meaning, so it meets the baseline for a well-documented schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all available BOJ statistical databases' with a specific verb and resource. It distinguishes from siblings which are about getting data code, layer, and metadata, making its role as a discovery tool unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Use this first to find the correct DB code before querying data,' providing clear when-to-use guidance. While no explicit when-not-to-use or alternatives are given, the sibling tools are for different purposes, so this is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• boj_get_metadataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that large databases may overwhelm responses and recommends using a small limit or layer-based queries. This addresses a key behavioral trait (response size), though it does not mention error handling or authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. First sentence defines the tool's output, second provides actionable guidance. Information is front-loaded and every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 params, no output schema), the description covers purpose, usage timing, and a critical caution about large databases. It does not detail output structure or error scenarios, but is sufficient for a metadata discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds value by explaining the purpose of parameters like db (database code) and limit (with specific guidance for large databases). It also mentions 'layer structure' in context, enriching the understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get metadata (series codes, names, units, frequencies, layer structure) for a BOJ database', specifying the verb and resource. It differentiates from siblings by indicating this is for discovering structure before fetching data, unlike boj_list_databases, boj_get_data_code, or boj_get_data_layer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using this tool to discover available series codes before fetching data, and includes a specific caution for large databases (e.g., CO/Tankan) to use small limits. It lacks explicit exclusion criteria or direct comparison with siblings, but the context implies when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.