Server Quality Checklist

Tool Scores

• list_directoryA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnly=true), description adds important behaviors: automatic ignoring of common directories, return structure (type, size, child count). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded purpose, no redundancy. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Describes return format (type, size, child count) despite no output schema. Mentions ignored directories. Lacks mention of ordering or potential performance issues with deep recursion, but otherwise sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions already cover all 4 parameters (100% coverage). Description repeats max_depth usage but adds no new semantics beyond schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists directory contents, distinguishes from sibling tools like read_content or edit_content. Mentions type field, recursion, ignored directories, and return fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use list_directory vs search_directory or other siblings. Only usage hints about max_depth recursion, no context on prerequisites or when to avoid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_directoryA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint=true) already disclose non-destructiveness. Description adds value by revealing automatic directory ignores (__pycache__, node_modules, .git) and performance implications of fuzzy matching. This goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each with distinct value: purpose, return format, filtering/ignores, and performance tip. Front-loaded with core purpose. No redundancy or irrelevant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 10 parameters (2 required) with full schema coverage, the description sufficiently covers usage context. It explains return format (grouped by file with line numbers and context) and automatic ignores, which are not in schema. No output schema exists, but the description fills the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. Description adds minimal parameter-specific info beyond the schema (e.g., include_pattern example, fuzzy performance note). These are helpful but not essential for understanding parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource scope: searches for a text pattern across all files in a directory. Distinguishes from sibling tools like search_content (likely single-file) and list_directory (listing vs searching). The description explicitly states the scope and output format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage guidance: recommends using include_pattern to filter extensions and defaults to fuzzy=False for performance. However, it does not explicitly tell when to prefer this tool over search_content or other alternatives, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_enclosingA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and non-destructive. The description adds valuable behavioral details: the depth parameter's effect and the fallback to a centered context window for unsupported languages or top-level code. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place. The first states the core purpose, the second provides usage details, and the third covers fallback behavior. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the purpose is clear, the description does not specify the return value format, and there is no output schema. For a tool that returns a 'complete enclosing definition', the agent would benefit from knowing what fields (e.g., name, type, line range) are included.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds meaningful guidance beyond the schema: it explains the use of depth=2 for parent definitions and describes context_lines as fallback-related. This enriches the agent's understanding of parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds the enclosing function or class for a specific line number, with a specific verb ('Find') and resource ('enclosing function or class'). It distinguishes from siblings like read_content by focusing on code structure rather than raw file content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (given a file and line), how to use depth for parent definitions, and the fallback behavior for unsupported languages. It does not explicitly explain when not to use this tool, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_overviewA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and destructiveHint=false. Description adds key behavioral details: uses Tree-sitter for code files, returns specific metrics, and clarifies it doesn't return file content. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  5 sentences, front-loaded with purpose, then usage, output, special behavior, and exclusion. No fluff; every sentence is informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 params and no output schema, it explains what the tool returns, its use case, and what it doesn't do. Combined with annotations, it's fully informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. Description adds context that the tool is for large files and provides usage guidance, but doesn't significantly enhance parameter meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Get' and resource 'file structure, size, and semantic outline'. Differentiates from siblings by explicitly saying 'Does NOT return file content - use read_content or search_content for that.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use FIRST when working with any file over 1000 lines or when you need to understand file structure.' Also provides alternatives for content retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• revert_editA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructive and not read-only. Description adds that current state is saved before reverting (making revert reversible) and returns a list of backups. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise 5-sentence description, front-loaded with purpose, no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains return behavior (list of backups with timestamps) and covers all relevant behavioral aspects for a revert operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The description adds value by explaining how to omit backup_id to use most recent and that the response includes backup list, going beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'restore' and resource 'file from automatic backups', with sufficient scope to distinguish it from sibling tools like edit_content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit context: 'Use when edit_content made unwanted changes'. Also explains backup creation and default behavior, but does not explicitly mention when not to use or alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_contentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes memory-efficient behavior, supports fuzzy/regex/inverted/count-only modes, mentions truncation of lines to 500 chars, and details the count-only return structure. Adds significant value beyond annotations (readOnlyHint, destructiveHint). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two-paragraph structure is efficient and front-loaded. First paragraph defines purpose and use cases; second paragraph details features and return behavior. Every sentence provides value, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters (all described in schema) and no output schema, the description covers return format with ranking, line numbers, context, truncation, and the count-only structure. Complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 9 parameters have descriptions in the input schema (100% coverage). The description adds context by summarizing key options (fuzzy by default, regex disables fuzzy) and explaining the special behavior of count_only. Provides meaningful guidance beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches large files for text patterns without loading entire content into memory. Lists specific use cases (functions, classes, errors, log entries) and differentiates from sibling tools like search_directory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use: 'Use when finding functions, classes, errors, log entries, or counting occurrences.' Provides clear context but doesn't explicitly state when not to use, though sibling names imply alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_contentA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (destructiveHint=true), the description adds that it creates automatic backups before changes, preview mode shows a diff without applying, and fuzzy matching handles whitespace/formatting differences. All disclosures are accurate and non-contradictory.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five sentences with front-loaded purpose, usage guidelines, and behavioral traits. Every sentence is informative and no redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no output schema, the description explains preview behavior and automatic backup but does not specify the return value when changes are applied. Minor gap, but otherwise comprehensive given annotations and schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters (100%), and the description adds contextual meaning: explains that changes are search/replace objects, preview mode shows diff, and fuzzy matching handles whitespace/formatting. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it edits large files using search/replace with fuzzy matching, distinguishing it from sibling tools like read_content (read-only) and revert_edit (undo).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using instead of line-based editing to avoid line number errors, notes that regex is not supported, and suggests using fuzzy=true for flexibility. Also mentions revert_edit as an alternative for undoing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_contentA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description goes beyond these by explaining the tool's behavior in detail: it reads files efficiently, supports multiple modes (lines, semantic, tail, head), and notes that pattern overrides offset. It also explicitly states it does not search, reinforcing its non-destructive nature. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at 4 sentences with no filler. It is front-loaded with the main purpose, then provides usage guidance, then mode details, and ends with a performance tip for large files. Every sentence earns its place, and the structure is logical and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no output schema, and no enums or nested objects, the description is fairly complete. It explains all modes, behavior for each, usage order, and a performance consideration. However, it does not describe the return format (e.g., raw text lines or structured data) or error handling (e.g., file not found). While the core clarity is high, a brief note on what the output looks like would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for all 5 parameters, so the schema already describes them. However, the description adds essential meaning: it explains that offset is 1-indexed, that pattern overrides offset, and that tail/head modes ignore offset. It also clarifies the purpose of each mode ('semantic' for tree-sitter, 'tail' for logs). These details are not in the schema's parameter descriptions, thus adding significant value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads specific portions of large files efficiently, using a specific verb 'read' and resource 'portions of large files'. It distinguishes itself from sibling tools like search_content by explicitly stating it does NOT search, and from edit_content by implication (read vs edit). The description also names the four modes, providing a clear picture of its functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Use after search_content locates content, or directly with tail/head modes for logs.' It also tells the agent what not to do: 'Does NOT search - use search_content first to find line numbers, then read_content to examine.' This clearly differentiates from the sibling search_content tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.