Server Quality Checklist

Tool Scores

• read_fileB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states the basic action and the file type (text), but does not disclose encoding, error handling, file size limits, or behavior for non-text files. No side effects are mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, composed of only two sentences with a brief Args block. Every word earns its place, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter) and has an output schema, so the description covers the core action. However, it lacks context on edge cases, error behavior, and alternatives. For a simple read tool, it is minimally adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds 'Relative path' to the parameter, which is more specific than the schema's 'File Path'. However, it does not elaborate on path constraints, existence requirements, or accepted formats. With schema coverage at 0%, this is partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Read the contents of a text file' uses a specific verb and resource, clearly distinguishing it from siblings like list_files, file_info, and search_files. It unambiguously states what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description does not mention any context, exclusions, or reference the sibling tools. The only hint is the tool name itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_infoA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose whether the operation is read-only, how errors are handled (e.g., missing file), or any side effects. The description only states the purpose, omitting behavioral traits beyond that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of one sentence for the purpose and a single line for the parameter. There is no fluff, and the 'Args:' structure is clear and front-loaded. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema present, the description covers the basic purpose and param. However, it lacks usage guidelines and any note on what metadata is returned or potential errors. It is minimally viable but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only lists 'file_path' with no description, so schema coverage is 0%. The description adds meaning by specifying 'Relative path to the file', which clarifies that the path is relative (not absolute) and is a file, not a directory. This is valuable semantic information that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Get metadata about a file' which is a specific verb+resource. It distinguishes this tool from siblings like list_files, read_file, and search_files, as it focuses on metadata rather than listing, reading content, or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention when to prefer file_info over list_files or read_file, nor does it give exclusions. This leaves the agent without explicit context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the search is recursive and starts from the current working directory, but omits important behavioral details such as whether hidden files are included, case sensitivity, or any side effects. While a search is intuitively read-only, the description does not explicitly confirm safety or resource implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded, with two sentences and an Args block that defines the only parameter. No unnecessary words or repetition; it efficiently conveys the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and an output schema, the description covers the fundamental operation and parameter semantics. However, it lacks details about return value structure or filtering behaviors, and does not position the tool relative to siblings. The context signal indicates an output schema exists, so return format may be documented elsewhere, but the description itself is thin.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds a clear explanation for the 'keyword' parameter: 'String to search for in filenames.' This goes beyond the bare schema definition, which only specifies the type as string. The phrase 'contain the keyword' clarifies that it is a substring match, though it does not specify case sensitivity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Recursively search for files whose names contain the keyword.' It specifies the target (files), the criterion (name contains keyword), and scope (recursive from current working directory). This distinguishes it from sibling tools like list_files or read_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given on when to use this tool versus alternatives. The description does not mention sibling tools or preconditions for usage, and only states the operational context of searching from the current directory. It does not explain scenarios where this tool is preferred over list_files or read_file.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states what it lists, but does not mention recursion behavior, hidden files, or error conditions. This leaves ambiguity about the tool's exact behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with an Args list, efficient and directly to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and an output schema. The description covers the essential purpose and parameter, though it lacks some behavioral details that would make it fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds that the 'directory' parameter is a 'Relative path' and that the default is 'root of workspace', which is more specific than the schema's default '.' and compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'files and directories' within a specified path in the workspace. It distinguishes from siblings like read_file (reads content) and search_files (searches), making it clear this is for enumeration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool over alternatives. It implies usage for listing directory contents but lacks explicit exclusions or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.