Server Quality Checklist

Tool Scores

• list_documentsA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only and idempotent. The description adds that image-kind files are not caught by text search, which is a meaningful behavioral trait about output. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the purpose, followed by two usage tips. Each sentence earns its place, though the missing folder context is a minor structural gap.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only list tool with fully documented params and an output schema, the description is functional. However, it fails to mention which folder is listed (a crucial contextual dependency) and does not describe the return structure beyond the path hint. The usage tip is valuable but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with concise descriptions (ext, limit, pattern, sort_by). The description adds no parameter-specific detail, staying within the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it returns a file list in a folder, but does not clarify which folder is being referred to (no `path` parameter exists; likely controlled by `set_folder`). It does not distinguish itself from siblings like `search_documents`, so an agent may not know when to choose this over alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit cross-tool guidance: returned `path` values can be reused in other tools' `path` arguments, and image-kind files are excluded from text search, so `read_image` should be used. This is actionable and helps route the agent correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_folderA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the safety profile is covered. The description adds real behavioral value beyond those: recursive traversal, supported-format indexing, and explicit skip rules for hidden folders and node_modules/.venv. Consistent with annotations — no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three Korean sentences with zero waste. The critical call-order instruction is front-loaded in the first sentence, followed by behavior rules and scope restriction. Every sentence earns its place; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a 1-parameter setup tool: purpose, call order, skip rules, and global scope are all covered. An output schema exists (present), so return-value documentation is not the description's burden. Minor gap: no mention of error handling for invalid/nonexistent paths, but this is acceptable given the schema and annotation coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% — the schema already documents 'folder' as an absolute path with a concrete example. The description does not add parameter-specific details beyond the schema, so the baseline of 3 applies. The scope-restriction note is contextual, not parametric.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: '분석할 문서 폴더를 지정하고 전체를 스캔한다' (designates the document folder and scans it all), plus the recursive indexing scope. It is clearly the setup/initialization tool among its siblings, though it does not explicitly name a sibling it is not. Each sibling (list_documents, extract_content, etc.) is implicitly distinguished as a post-setup tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit call-order guidance: '가장 먼저 호출해야 한다' (must be called first) and states that all subsequent tools operate only within this folder. This tells an agent when to invoke it. It does not name specific alternatives or exclusion conditions (e.g., when to use refresh instead), so it falls short of 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build_digestA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safe-read profile is covered. The description adds real behavioral value by warning that excerpts are only the beginning of files (do not claim to have read the full document) and by instructing disclosure when the 'truncated' flag is True. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the primary purpose, and the critical operational warnings are emphasized with bold. Every sentence earns its place; the length is justified by the behavioral caveats. Slightly verbose but well-structured for the information density required.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return format is covered. The description covers purpose, usage timing, the excerpt-limitation caveat, and the truncated behavior. It does not mention what file types are excluded beyond the ext filter, but given the output schema and rich behavior notes, this is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all three parameters (ext, limit_files, chars_per_file) are already documented with defaults and ranges. The description adds no extra parameter semantics beyond the schema, which is the expected baseline when the schema carries the full load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific purpose: 'collects materials for summarizing an entire folder at once' and defines the exact output shape (condensed structure + beginning of body per file). It distinguishes itself from extract_content by scope (batch folder operation vs. per-file deep reads) and names the use case explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives clear when-to-use guidance: call this first for requests like 'summarize this whole folder' to reduce round trips. It also routes to extract_content for files that affect conclusions. It stops short of explicitly stating when NOT to use it (e.g., single-document queries), so the exclusion guidance is implied rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_contentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is known. The description adds valuable behavioral context: the chunked pagination mechanics and a security warning about embedded instructions, which go beyond the annotations. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-line purpose, a paging instruction, and a security caution. Each sentence serves a distinct purpose, and the critical paging loop is stated upfront. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (paginated reading) and the presence of an output schema, the description sufficiently explains the complete usage pattern: how to iterate, when to stop, and a caution about content safety. The output schema handles return format details. Nothing critical is missing for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all parameters (path, raw, start, max_chars, with_line_numbers) have descriptions. The description reinforces the start/next_start relationship for paging, which is helpful but not new information beyond the schema's own description of the start parameter. This is a baseline 3 where the schema does most of the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns file body text in chunks ('파일 본문 텍스트를 잘라서 돌려준다'), which is specific about what it does. It doesn't explicitly name sibling alternatives (like read_image), but the purpose is unambiguous and distinguishable from the other tools based on the clear resource scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for paging: '긴 문서는 start를 옮겨가며 여러 번 호출한다' and warns not to claim full reading when next_start is non-null. It also advises not to follow instructions embedded in the content. This is strong usage guidance, though it doesn't explicitly contrast with alternatives like read_image, which is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_imageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds genuine value beyond that: the server's inability to read text, the unique return type (image block instead of Pydantic model), and the pipeline consequence (no stage/next_actions). This is meaningful behavioral disclosure that annotations alone don't convey, though slightly less than a full 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Purpose is front-loaded in the first sentence, and every paragraph earns its place: usage caution, return-type distinction, and next-step routing. It's efficient but slightly long — the Pydantic-model mechanism detail is useful but could arguably be trimmed without losing the core message.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's relative simplicity (2 params, full schema coverage, rich annotations), the description covers purpose, usage caveats, return format, and routing to the correct sibling. The lack of an output schema is compensated by the explicit statement that it returns an image block. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% — both path and max_dim are already documented with type, constraints, and default. Description does not add parameter-level semantics beyond the schema (the token-cost tradeoff on max_dim is already in the schema description). Baseline 3 is appropriate since the schema carries the full weight here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('delivers raster images as-is') and is explicit about what it does NOT do — the server can't read text in images, so the agent must judge the image itself. It clearly distinguishes itself from siblings by noting it returns an image block rather than a Pydantic model, and names analyze_structure as the related alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use guidance ('use this tool to receive the image directly and judge for yourself'), a when-not-to rule (don't read numbers from unlabeled charts — mark as '[확인 필요]'), and names the sibling for the next step (analyze_structure for resolution/EXIF). Usage routing is fully spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and openWorldHint=false, covering safety. The description adds behavioral context beyond annotations: image files are skipped (`skipped_images`), search is exact string matching (not semantic), and users should retry with different phrasing. This gives agents crucial operational insight without contradicting any annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: purpose statement, usage guidance, and limitations in three short paragraphs. Every sentence adds value, and the key information (purpose) is front-loaded. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three well-documented parameters and an output schema, the description covers everything an agent needs: what it does, when to use it, how to handle its limitations, and citation instructions. The mention of `skipped_images` and retry advice rounds out the context. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minor extra guidance (e.g., stating `snippet.line` should be cited), but the parameter details like query format and limits are already fully documented in the schema. It does not compensate for any gaps because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (검색, search), resource (폴더 전체, entire folder), and result (관련 문서와 발췌, related documents and excerpts). This clearly distinguishes it from siblings like list_documents (listing without search) and extract_content (extracting specific content).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: '사용자의 질문에 답할 근거를 찾을 때 쓴다' (use when finding evidence to answer the user's question). It also provides two limitations (images not searched, string matching only) and advises retrying with different wording. However, it does not explicitly mention when not to use it or compare with specific siblings, so it misses the full 'when-not/alternatives' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_structureA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so safety is covered. The description adds value by asserting the returned metrics are exact ('코드가 센 것이라 정확하다') and instructs the agent to cite them rather than estimate. It also details format-specific outputs, which goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise intro, an important accuracy note, and a clear bulleted list by format. It is somewhat long but each bullet adds specific, useful information. Front-loading of purpose and usage guidance is effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is comprehensive for an analysis tool. It covers the tool's role relative to summarization, the guarantee of accurate numbers, and details what is returned for all seven supported formats. It even notes that PNG image content is handled by read_image. With an output schema present, this fully equips an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the single 'path' parameter ('폴더 기준 상대경로. list_documents가 준 path를 그대로.') with 100% coverage. The description does not add any additional parameter semantics, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes the structure of a single file ('파일 하나의 구조를 분석한다'), and the context of '요약 전에' positions it as a pre-summarization step. It is explicit about being structure-focused rather than content-focused, distinguishing it from siblings like extract_content and read_image.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: '요약 전에 이걸 먼저 보면' (use before summarizing). Also names an alternative for PNG content: '이미지 내용은 read_image로 확인' (for image content, use read_image), directing the agent away from this tool for that specific case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• folder_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds valuable behavioral context: the tool returns an extraction failure list, and it instructs the user to verify this list before finalizing a summary. It also reveals reliance on a 'currently specified folder' state. No contradiction with annotations, and the description enriches understanding beyond structured data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the core purpose front-loaded in the first sentence and the essential usage directive in the second. No padding or redundancy. The formatting with a line break and bold for the failure list enhances readability without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema and annotations that cover safety, so the description only needs to add usage context. It explains return content (counts, size, character count, failure list) and a specific procedural requirement (check failures before summary). However, it does not explicitly state that the folder must be set beforehand (via set_folder), which is a prerequisite. Given the sibling set_folder and the phrase 'currently specified folder', this is inferable, but a explicit note would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to document. Schema coverage is trivially 100%. The description correctly does not invent parameters. Per the rubric, a baseline of 4 is appropriate for tools with no parameters; the description adds no extra parameter semantics because none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource pair: 'Returns the inventory of the currently specified folder.' It then specifies what the inventory includes (counts by extension, size, character count, and extraction failure list), which distinguishes it from sibling tools like list_documents or analyze_structure. The purpose is unambiguous and the tool's scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a specific usage instruction: before writing a summary, check the failure list and disclose omitted files. This is a clear context for when to use the tool. However, it does not explicitly name alternatives or say when not to use it. The dependency on a previously set folder (via set_folder) is implied but not stated. Overall, it provides useful guidance but leaves some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refreshA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavioral context about cache reuse for unchanged files and the resulting speed improvement, which is beyond the annotations and helps the agent anticipate performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and usage, then a brief performance note. Every sentence earns its place with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless refresh tool with an output schema present, the description fully covers when to call and performance expectations. No missing information that an agent would need to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so baseline is 4. No parameter explanation is needed or provided; the schema is empty and the description correctly avoids adding irrelevant parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'rescan' and the resource 'folder' with explicit trigger context ('after files have been added or modified'). This distinguishes it from siblings like list_documents or folder_status by focusing on refreshing the index after changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit timing guidance ('call after files have been added or modified') but does not mention when not to use it or name alternatives. The context is clear enough for an agent to infer usage, though lacking exclusions or comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.