Server Quality Checklist

Tool Scores

• file_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'search for files' without revealing what is returned (paths, matches, counts), whether the search is recursive, how multiple criteria are combined (AND/OR), or any performance implications. This is insufficient for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the primary action and resource, then lists criteria. There is no wasted verbiage. It could be slightly clearer, but it earns high marks for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 8 parameters, no output schema, and no annotations, the description is minimal. It leaves critical questions unanswered: return format, recursion behavior, combination logic, and edge cases. An agent would need to make assumptions or probe the tool to use it correctly. This is a significant gap for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (every parameter has a description), so baseline is 3. The description adds some value by grouping criteria into 'size range' and 'date range', which implies relationships between min/max_size and modified_before/after. It also provides glob/regex context for name_pattern and content_pattern. However, it does not explain how parameters interact or any defaults beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (search) and resource (files) and lists the search criteria (name pattern, content, size, date). It distinguishes itself from sibling tools by implying a search operation versus listing (dir_tree), statistics (file_stats), duplicates (duplicate_finder), or counting (code_counter). However, it does not explicitly name alternatives, so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus its siblings. It does not mention exclusions, prerequisites, or when not to use it. The description implies search use cases but lacks explicit context for selecting this tool over alternatives like dir_tree or file_stats.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dir_treeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It mentions the output format ('visual directory tree') and configurability, but does not state side effects, permissions, performance implications, or error handling. It does not clarify that this is a read-only operation, which is a notable omission without annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence. It front-loads the core purpose ('Generate a visual directory tree structure') and then immediately provides the key capabilities. There is zero unnecessary verbosity, making it easy for an agent to grasp quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with full schema coverage and no output schema, the description is adequate but not complete. It explains what it does but omits details like the exact format of the output (e.g., ASCII art, indentation), whether it follows symlinks, or how it handles permission issues. These are minor for a tree tool, but with no annotations to fill the gaps, it remains at a middle score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all three parameters with descriptions (100% coverage), setting a baseline of 3. The description adds a little context by mentioning 'configurable depth and ignore patterns', which maps to max_depth and ignore_patterns, but it adds no extra meaning beyond the schema. It does not compensate for any gaps, so it stays at baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the function: generate a visual directory tree structure, and it names the analogous `tree` command. It specifies the verb ('Generate') and resource ('directory tree structure') and mentions configurable depth and ignore patterns. However, it does not explicitly differentiate from sibling tools, so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like file_search or file_stats. The description only says what it does, not the context in which it is preferred or when it should be avoided. No exclusions or alternatives are mentioned, offering no decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• code_counterA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It names the four line types and per-language grouping but does not mention recursion behavior, file inclusion/exclusion rules, binary file handling, or output format. These are significant behavioral details for a directory-traversing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. The 'like cloc/scc' parenthetical adds a compact orienting reference without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description names the output metrics and provides a known reference tool, which is helpful. However, it omits default traversal depth behavior, whether directories are recursed by default, and the shape of the returned result, leaving an agent with some open questions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so dir_path and max_depth are already documented in the schema. The tool description adds no parameter-level semantics, so the baseline of 3 applies—the schema does the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Count lines of code by programming language' and enumerates the exact metrics (total, code, comment, blank lines). The cloc/scc reference further clarifies the intent and clearly separates this tool from sibling file tools like dir_tree and file_stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool—when line-count statistics are needed—but it does not explicitly contrast this with dir_tree, file_stats, or file_search. There is no when-not-to-use statement, so the agent must infer the appropriate context from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_statsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description lists the statistics produced and implies a read-only analysis, but with no annotations it does not disclose traversal behavior, handling of permission errors, or whether the analysis is recursive. The max_depth parameter hints at recursion, but the description itself leaves that implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence with a colon-separated list. The action and target are front-loaded, and every listed item adds useful information without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is low-complexity and the schema covers both parameters, so the listed output types are helpful. However, with no annotations and no output schema, the description could still say more about return shape, recursion behavior, or failure conditions to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both dir_path and max_depth. The description does not add extra parameter meaning beyond what the schema provides, which meets the baseline for a well-covered schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Analyze a directory') and enumerates concrete outputs: total files, total size, file count by extension, largest files, and newest/oldest files. This distinguishes it from sibling tools like dir_tree or file_search, which serve different structural or search purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus its siblings, nor are any alternatives mentioned. The only implied usage is analyzing a directory, but no exclusions or selection conditions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• duplicate_finderA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does disclose the algorithm (size + MD5 comparison) and the return structure (groups of duplicates with paths and wasted space). However, it does not state that the operation is read-only/safe, mention recursion or symlink behavior beyond the max_depth default, or warn about the performance cost of hashing many files. Meaningful disclosure, but with zero annotation coverage, these gaps matter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero filler. The first sentence front-loads purpose and method; the second discloses the return format. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-param tool with full schema coverage and clear purpose, the description covers the core needs: what it does, how, and what it returns. The gaps are the missing safety declaration (read-only is implied but not stated, and there are no annotations), absence of performance/limitation warnings, and no explicit sibling differentiation. Adequate but with clear gaps relative to its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents dir_path, min_size, and max_depth. The description references the 'size' comparison, which loosely contextualizes min_size as a pre-hash filter, but adds little beyond what the schema already states. Baseline 3 applies when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Find'), a resource ('duplicate files'), and the exact method used ('comparing size and MD5 hash'). This clearly distinguishes it from siblings like file_search (searching files) and code_counter (counting code), so an agent can tell them apart without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies the use case (identify duplicates in a directory), and the scope is directional rather than explicit. It does not name alternatives or state when not to use this tool vs file_stats or file_search, leaving the routing decision to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.