Server Quality Checklist

Tool Scores

• analyze_folder_structureA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive. The description adds crucial behavioral detail beyond that: it explains that the tree includes only up to max_files entries with list_truncated flag, while statistics are always based on the full file set. It also details the default extension set when no filter is given. No contradiction with annotations; this transparency is valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized: opens with the core purpose, then a usage hint, a Returns section, and examples. Each section contributes to understanding, though it is slightly longer than strictly necessary. The structure is front-loaded with the main purpose and comparison, making it easy to skim. Optimally concise for the complexity it covers.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and safety annotations provided, the description still adds essential behavioral details (truncation semantics, stats being global) and usage examples. It covers the necessary context for correct invocation, including how max_files affects results and the default extension pool. No missing critical information for typical usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter already having detailed descriptions (folder_path as absolute path, max_files with truncation explanation, extensions with default set). The tool description does not add supplementary meaning beyond the schema; it neither repeats nor enhances parameter semantics, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it analyzes the folder's subdirectory tree, extension statistics, size, and largest files, with a specific verb and resource. It distinguishes itself from scan_folder by noting it goes one level deeper into structure. However, it does not differentiate from the similarly named sibling analyze_folder, which could create ambiguity about which tool to use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: use it when you need an understanding of folder organization (subfolder distribution, largest files) and explicitly says not to use it when you only need a quick file list, directing to scan_folder. However, it omits any comparison with analyze_folder, another sibling that might be a relevant alternative, leaving some usage ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_image_textA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, destructiveHint), the description discloses important behavioral traits: it requires Tesseract-OCR, it is OCR not vision (so it does not understand image meaning), and it may return empty results for text-less images, with next_actions explaining why. This adds significant value over annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by usage guidelines, return details, and examples. It is slightly verbose with the examples, but each section carries useful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple OCR tool, the description covers purpose, usage, limitations, and dependencies. The output schema is mentioned (though not shown) and the max_chars truncation is addressed. The only minor gap is that it could specify what happens on file-not-found or unsupported formats, but these are likely covered elsewhere.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters clearly described (file_path as absolute path, max_chars as truncation limit). The description itself does not add further parameter detail, so per the calibration baseline of 3 for high coverage, this score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the exact verb and resource: 'Reads a PNG image with OCR to extract text' and explicitly distinguishes it from read_document for document formats. It clearly states the tool's scope and what it does not do.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('for screenshots, scanned notes, labels on diagrams') and when not to use it ('for pdf/docx/pptx/svg documents → read_document'), providing clear exclusions and routing to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_folderA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint as safe. The description goes beyond by revealing that it is a batch tool that continues processing on per-file errors, transparently reports skipped files via skipped_due_to_limit, and returns statuses (OK/ERROR/PARTIAL). This adds significant behavioral context not present in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: main purpose first, followed by batch/error/limit behavior, then return format, then examples. It is somewhat lengthy but every section adds value, including explicit '사용' and '사용하지 않음' examples. The front-loading of the core purpose is effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, output schema present), the description covers all essential aspects: batch semantics, failure handling, limit transparency, return structure, and usage alternatives. It also includes practical example prompts. Nothing an agent needs to correctly invoke the tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides complete descriptions for all 5 parameters (100% coverage), so the baseline is 3. The description does not add new meaning to parameters beyond repeating max_files' skipped_due_to_limit behavior, which is already in the schema. No additional semantic value is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts all target files (pdf/docx/pptx/svg/png) in a folder and returns a report. It explicitly distinguishes itself from siblings: scan_folder (file list only) and read_document/read_image_text (single file). The verb '추출' and resource '폴더' make the purpose specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool (batch extraction, folder document summarization) and when not to use it, naming the alternatives: scan_folder for file lists and read_document/read_image_text for individual files. It also clarifies behavior under max_files limit and partial failures, leaving no ambiguity about invocation context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_documentA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint. The description adds valuable behavioral context: it returns the original text (possibly truncated), and explains the truncation contract (increase max_chars to retrieve the rest). This goes beyond annotations and clarifies the tool's output semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear statement of purpose, usage context, return details, and examples. It is slightly lengthy but every section earns its place, and the core purpose is front-loaded. Minor redundancy with the annotation 'readOnlyHint' doesn't detract significantly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and the tool is simple, the description covers all necessary aspects: what it does, when to use it, how output is truncated, and how to recover full text. The examples reinforce correct usage. Nothing critical is missing for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are fully documented. The description reinforces the purpose of max_chars in the truncation note but doesn't add new semantic details beyond the schema. Baseline 3 is appropriate when the schema carries the full parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts text from document files (pdf/docx/pptx/svg) and explicitly notes it does not summarize. It differentiates from siblings by naming the specific use case (inspecting files from scan_folder/analyze_folder) and alternative tools (read_image_text, analyze_folder), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (detailed look at a found file, or continuing a truncated analyze_folder summary) and when-not-to-use (PNG images -> read_image_text, folder-wide scanning -> analyze_folder). Also gives a concrete example of user phrasing, leaving no ambiguity about when this tool should be selected.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_folderA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations already declare readOnlyHint=true and idempotentHint=true, the description adds crucial behavioral context beyond this: it explicitly states that document contents are NOT read, only metadata (path/size/modified date). It also discloses truncation behavior (list_truncated flag, by_extension always full). These details inform an agent about side-effects and guarantees that annotations alone do not convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a clear first sentence stating the core purpose, followed by a behavioral note, return format details, and usage examples. It is slightly verbose (multiple paragraphs) but every section earns its place, front-loading the key action and scoping. It could be tightened, but structure is strong.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, output schema, multiple siblings), the description covers all essential aspects: what it returns (files array fields and by_extension counts), truncation edge-case behavior, and guidance on when to use alternatives. With an output schema present, the description need not detail return types, yet it already does so sufficiently. No critical information is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage, so the baseline is 3. The description adds marginal value by reinforcing that max_files truncates the list but not the by_extension counts, which is already implied in the schema's max_files description. Since the schema documents all parameters fully, the description does not meaningfully augment parameter semantics beyond what is already structured.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it lists target files (pdf/docx/pptx/svg/png) in a folder and provides counts by extension. It explicitly distinguishes itself from siblings by stating it reads only metadata, not content, and names specific alternatives (analyze_folder, analyze_folder_structure) with conditions for their use. This makes the tool's purpose unmistakable and differentiates it from the four siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: use when you want to quickly check which documents exist and how many there are, with example queries. It also gives clear exclusions: do not use for content summarization (use analyze_folder) or for subfolder structure/statistics (use analyze_folder_structure). These concrete when-to-use and when-not-to-use instructions leave no ambiguity for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.