Server Quality Checklist

Tool Scores

• run_qa_singleA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses that the browser is shown by default, but it does not explain side effects, output/return format, error behavior, permissions, or whether the run mutates state. This is minimal information for an execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences that lead with the core action, then add purpose and a key behavioral note. Every sentence earns its place with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description fails to tell the agent what the tool returns (e.g., pass/fail, logs), what prerequisites exist (e.g., the item must be prepared), or whether the run has side effects. The simple parameters do not compensate for these critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides full descriptions for both item_id and headless, so the coverage baseline is 3. The description does not add significant parameter-level detail beyond restating the headless default, which is already in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Runs') and resource ('only one specific item'), clearly distinguishing this from the sibling tool run_qa which likely runs the full suite. The debugging/verification context further clarifies its niche.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly frames the tool for debugging and confirmation purposes, giving a clear context for when to use it. However, it does not explicitly name alternatives or state when not to use it (e.g., for batch runs, use run_qa instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_qa_itemA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the main behavior (accessing a page, collecting DOM) and notes a dependency. However, it does not state potential side effects (e.g., whether it modifies item status), authorization requirements, or long-running behavior, leaving uncertainty about operational impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with clear structure. The first states the action, the second explains return value and a key prerequisite. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers the main action, return purpose, and a critical precondition. It could be more complete by specifying what the returned data includes, but it is adequate for an agent to decide usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the single parameter is 100% with a clear description ('Item ID to prepare'). The description itself does not add parameter information, but the schema suffices, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('accesses target page', 'collects DOM structure') and names the resource ('specific item'). It clearly distinguishes from sibling tools like list_qa_items or save_qa_script by stating its role in script generation data collection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it is used before script generation and explicitly states a prerequisite (preceding scripts must be ready for intermediate items). However, it does not name alternative tools or specify when not to use it, so it falls short of 'explicit alternatives' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• register_qa_sheetA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses key behavioral traits: it parses Excel, registers items, avoids DOM collection, stores only metadata, and returns groups/independent_items classification. However, it does not mention potential side effects like overwriting existing items, required permissions, or error/edge-case behavior, leaving some ambiguity for a mutation-like operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, efficient and front-loaded. It states the core action first, then adds a meaningful constraint and return-value note without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description is reasonably complete. It covers what the tool does, what it does not do, and what it returns. The main gap is the lack of detail about error behavior or post-conditions, but this is acceptable for the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully documents the only parameter (sheet_path) with a clear description ('Excel(.xlsx) 파일의 절대 경로'). The tool description adds minimal extra meaning beyond the schema, so the baseline of 3 is appropriate given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the exact verb and resource: parsing an Excel .xlsx inspection sheet and registering its items. It clearly distinguishes from siblings by explicitly stating it does not perform DOM collection, and clarifies its scope as metadata-only storage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (register items from an inspection sheet) and an explicit exclusion: 'DOM 수집은 하지 않고' (does not perform DOM collection). It does not name alternative sibling tools, but the usage boundaries are reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_qa_itemA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It reveals a significant side effect: changing description or path resets scriptStatus to pending, requiring script regeneration. This adds valuable context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two sentences that front-load the main action and then add a crucial side-effect note. No wasted words, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an update tool with 7 parameters and no output schema, the description covers the core purpose and a key behavioral consequence. It does not explicitly state partial-update semantics, but the schema's optional fields imply that, and the description is sufficient for an agent to understand the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds extra meaning for two parameters (description and path) by explaining their side effect on scriptStatus, going beyond the schema's simple field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states '특정 검수 항목의 메타데이터를 수정합니다' (modifies metadata of a specific QA item), providing a clear verb and resource. This distinguishes it from sibling tools like delete_qa_item or save_qa_script.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for updating QA item metadata, giving clear context but no explicit when-to-use/when-not-to-use guidance or references to alternatives. It does not mention other tools for related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_qa_itemsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It notably discloses that the tool automatically syncs the 'data/sheets/' folder before retrieval ('조회 전 data/sheets/ 폴더를 자동 동기화합니다'), which is a significant side effect not evident from the schema. This goes beyond basic purpose and gives the agent awareness of pre-conditions. However, it does not mention the return format or potential read-only nature, but for a simple list tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, directly states the primary purpose first, then adds the key behavioral note about auto-sync. Every word is functional, with no fluff or repetition. It is appropriately front-loaded and compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 optional string params, no nested objects, no output schema), the description is complete: it explains the main action, the filters, and a notable pre-execution side effect. The absence of return-format details is acceptable since no output schema exists and the purpose is straightforward. It fully equips an AI agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: both 'site' and 'account' have descriptions in the schema ('특정 사이트 키로 필터 (생략 시 전체)'). The description mentions filtering by site and account but adds no additional meaning beyond what the schema already provides. It does not explain formats, value constraints, or relationships between parameters, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: '등록된 검수 항목 목록을 조회합니다' (retrieves the list of registered QA items). It specifies the resource (검수 항목/QA items) and the action (조회/retrieve list), and mentions optional filters (site, account). This distinguishes it from sibling tools like register_qa_sheet, update_qa_item, and run_qa, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: it is a list operation with optional site and account filters. It does not explicitly state when to use this vs alternatives, but the nature of the tool (listing) is self-evident and distinct from siblings. There are no exclusions or alternatives mentioned, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_qa_itemA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses that deletion also affects 'related script files', a significant side effect beyond just the QA item. It also reveals the constraint about Excel-registered items and the support for single/batch modes. However, it leaves out details like irreversibility, permissions, or return values, but the key behavioral traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the action and modes, the second states the important limitation. It is front-loaded and contains no redundant words, earning a high score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a delete tool with no output schema and only two parameters, the description is sufficiently complete: it covers scope, supported modes, and the critical Excel restriction. Given no annotations, it effectively carries the necessary context for an agent to use the tool safely, though it could mention prerequisites like item existence.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents item_id and item_ids with clear Korean descriptions (100% coverage). The description restates the single/batch distinction but adds no new information about parameter types, constraints, or mutual exclusivity beyond what the schema provides. Therefore, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deletes QA items and related script files, with specific verb '삭제' (delete) and resource '검수 항목' (QA item). It also distinguishes from siblings by covering both single and batch deletion, and complements other QA tools like list/update/run. This makes it unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a when-not-to-use rule: items registered in data/sheets/ Excel cannot be deleted directly and must be removed via Excel. This exclusion guides the agent to choose an alternative workflow for Excel-registered items, and supports both single and batch cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_qaA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the output format (text summary) and the side effect of saving failure screenshots to a specific directory. It doesn't mention other potential side effects like auth or data mutation, but for a QA run tool, the disclosed behavior is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a distinct purpose: stating the main function, describing results and storage, and directing to an alternative. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks an output schema but mentions the return type. It explains the batch execution purpose, output format, and side effect location, which is reasonably complete for a batch tool. It doesn't detail failure interpretation or prerequisites, but sibling tools likely cover setup, so it is still adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers 100% of the parameters with descriptions, so the description adds little semantic value beyond the schema. It implies 'all or some' which aligns with the schema but doesn't elaborate on parameter details, so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool executes all or some registered scripts for QA inspection, using a specific verb '실행' (execute) and a resource. It also explicitly distinguishes itself from the sibling tool run_qa_single by positioning itself as a batch runner, which helps with selection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use this tool (for batch execution of multiple scripts) and directs users to run_qa_single for individual failure screenshots, thus naming an alternative and giving a clear when-not scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_qa_templateA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly reveals the important side effect: if the template does not exist, it will be automatically created. This is a key behavioral trait beyond a simple read. It does not elaborate on permissions or return format, but for a simple tool, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary action ('returns the file path') and then a conditional side effect. Every word adds value with no redundancy or irrelevant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and no annotations, the description fully covers the essential behavior: it returns a path and self-heals by creating the template if missing. This is complete for an AI agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is vacuously 100%. The description does not need to explain parameter semantics, and the baseline for 0 params is 4. It adds context by explaining the tool's fallback behavior, which is more than the empty schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it returns the file path of the QA sheet Excel template. It also notes that it auto-creates the template if missing, adding specific resource and behavior. This distinguishes it from sibling tools like 'create' or 'register' operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: when you need the QA template path, it provides it, and handles missing templates by creating them. While no explicit alternatives are mentioned, the tool's purpose is simple and there is no competing sibling for retrieving a template path. It provides adequate context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_qa_scriptA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the tool's behavior: it performs syntax validation, returns error messages on failure, and enforces specific script constraints (CommonJS export, single page argument, no external imports, required return object). It also clarifies that the first page navigation is handled by the runner, adding important behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured: it starts with a one-sentence purpose, then breaks conventions into bullet points, and includes a concrete example. Every section contributes necessary constraints and guidance, making the length justified rather than wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (script validation with many rules) and lack of an output schema, the description is highly complete. It covers the full script contract, failure behavior, and provides a working example. Nothing critical is missing for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema describes both parameters, the description adds significant meaning to script_code by specifying the exact expected format, return shape, and forbidden patterns. This goes well beyond the schema's one-line description and is essential for correct usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states '에이전트가 생성한 Playwright 스크립트를 구문 검증하고 저장합니다' (validates and saves the agent-generated Playwright script), using a specific verb and resource. It is easily distinguished from sibling tools like register_qa_sheet or run_qa, as it focuses on saving and validating a script.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit '스크립트 작성 규약' section with detailed rules on export format, arguments, return value, and prohibited imports. It also explains that on validation failure an error message is returned so the script can be fixed and retried, giving clear context for when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.