Server Quality Checklist

Tool Scores

• grepA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return format ('matching lines with file paths and line numbers') but does not explicitly state read-only behavior or other safety traits. The verb 'Search' implies read-only, but it could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, then covering filters and return format. No wasted words, clear and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with 4 parameters all documented. The description covers the main function, filtering options, and return format. It doesn't mention edge cases or performance, but for a search tool it is sufficiently complete given the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and each parameter has a thorough description. The tool description adds 'Supports regex patterns and optional glob filtering,' which reiterates schema fields without adding new semantics. Baseline 3 applies for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search for text patterns in files under the root' – a specific verb and resource. It distinguishes from siblings like read_file (reading a specific file) and explore_filesystem (browsing structure) by focusing on content search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Supports regex patterns and optional glob filtering to limit which files are searched,' providing clear usage context. However, it does not explicitly contrast with alternatives or state when not to use it, though the context is sufficient for most cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_fileA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It covers destructive edge cases: non-empty directory handling and protection against deleting the root. However, it omits whether deletion is permanent or any permission requirements, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences, no fluff, and directly addresses key aspects. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers all essential behaviors: what can be deleted (file or directory), the recursive nuance, and root safety. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters well. The description adds value by clarifying the root-relative scope and root-deletion refusal, but these are complementary details rather than extensive new semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Delete a file, or a directory, under the root' with a specific verb and object, and distinguishes itself from sibling tools like read_file and edit_file by its delete action. The root-scoping detail further refines its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides important context such as 'Deleting a non-empty directory requires recursive: true' and that it refuses to delete the root, but it does not explicitly contrast with alternative tools or state when not to use it. Usage is implied rather than directly compared to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the output format (line numbers prefixed, total line count), the inclusive nature of the line range, and the scoping to 'under the root.' This exceeds a minimal disclosure, though it omits details like encoding or error handling, which are not critical for this simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose and then the optional scoping and return format. Zero fluff, every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with no output schema, the description covers purpose, parameters, and return format (line numbers and total count), which is sufficient for an agent to invoke it correctly. It also notes the root constraint, covering the main operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description restates that startLine/endLine are 1-indexed and inclusive and mentions defaults, but these are already fully described in the schema. No additional meaning is added beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Read the text content of a file under the root' with a specific verb and resource, and clearly distinguishes from siblings like edit_file (modify), grep (search), and explore_filesystem (browse). It also specifies the optional line-range feature, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: it's for reading a file, and the line-range option is explained. However, there's no explicit guidance on when not to use it (e.g., for large files or searching) or any mention of alternatives like grep for content search or explore_filesystem for directory listing, leaving the distinction to the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explore_filesystemA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It usefully clarifies that root means the server's start directory and that globs support *, **, and ?. However, it omits traversal behavior (recursive vs. shallow), whether results include metadata like paths or sizes, and how directories are distinguished from files—gaps the agent must guess at.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly packed sentences: the first states the core action and location scope, the second covers both optional parameters in one clause. Every word contributes—no filler, no restating of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity listing tool with no output schema and no annotations, the description covers the essential aspects: what is listed, the base directory semantics, and the scoping/filter options. The main gap is the absence of return-value shape (absolute vs. relative paths, ordering), which the agent must infer. Minor but real, keeping it below a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds genuine value by specifying the root base for subpath resolution ('relative to root') and documenting the supported glob syntax (*, **, ?), which goes beyond the schema's brief examples. This enrichment justifies the 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  "List files and folders under the server's root directory" uses a specific verb (list) plus a precise resource scope (files/folders under the server's start directory). This clearly distinguishes it from content-focused siblings like read_file, grep, and edit_file, which all operate on the contents of individual files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use it as a discovery tool ('List files and folders') and offers optional scoping ('scope to a subpath and/or filter with a glob pattern'). However, it never explicitly states what it is NOT for (e.g., reading file contents, searching inside files) or names alternatives, leaving the contrast with grep/read_file implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_fileA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly discloses key behaviors: creates a new file under root, creates missing directories, fails if file exists unless overwrite is true. This is strong transparency, though it omits details like permissions or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the main purpose, followed by two short sentences for key details (directory creation and overwrite). No fluff; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple CRUD create operation with 3 params and no output schema, the description covers the essential behavior. It explains path resolution, directory creation, and overwrite, which are key for correct invocation. It could include a note about what happens on success (e.g., returns nothing), but that is minor given the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all parameters described in schema), so baseline is 3. The description adds value by clarifying that 'path' is relative to root (schema says relative, but description emphasizes it) and that content is for text. It also explains overwrite default behavior in context, which goes beyond schema's default value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create a new file'), the target resource ('under the root'), and the content parameter ('with the given text content'). It also distinguishes from siblings by specifying the creation action and outlining key behaviors like creating missing directories and overwrite semantics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use: to create a new file, and it differentiates from edit_file and delete_file by focusing on creation. It also explains the overwrite behavior, which is a common decision point. However, it does not explicitly state when not to use (e.g., use edit_file for existing files unless overwriting).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_fileA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It thoroughly explains the edit semantics (inclusive line ranges), the bottom-to-top application order for multiple edits, and the empty array for deletion. It also covers the insert case via endLine = startLine - 1, and mentions the createIfMissing flag implicitly through schema, but the description's core behaviors are well-disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the verb and object, and every sentence adds essential operational detail (line indexing, insert/delete patterns, ordering). No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 params, no output schema, and no annotations, it covers all necessary behavior: line ranges, deletion, insertion, ordering, and path restriction. The createIfMissing param is in schema but not described, but its default false and the tool's purpose make it low-risk. The description is complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, so the description must compensate. It does: it explains each parameter's role (startLine, endLine, newLines) and the special values for insert/delete. It also clarifies the semantics of multiple edits.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it applies git-style line edits to a file, distinguishing it from create/delete operations. It specifies the file must be under the root, and the exact semantics of line replacement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for editing existing files, and explicitly explains deletion and insertion patterns. It does not explicitly state when not to use it (e.g., for creating new files, use create_file), but the contrast with sibling tools is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.