Server Quality Checklist

Tool Scores

• get_file_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Get detailed information') but doesn't specify what information is returned (e.g., metadata, permissions, size), whether it's read-only (implied but not stated), or any constraints like rate limits or authentication needs. This leaves significant gaps for a tool with no structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It could be slightly more structured by hinting at return values, but it earns its place by being direct and clear, with no wasted verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage, the description is minimally adequate. However, with no output schema and no annotations, it should ideally explain what 'detailed information' includes to compensate. It meets the basic threshold but lacks depth for full contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'path' parameter clearly documented. The description adds no additional parameter details beyond what the schema provides, such as path format examples or constraints. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('detailed information about a file or directory'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_directory' (which might list contents) or 'search_files' (which might search across files), so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_directory' or 'search_files'. It doesn't mention prerequisites, such as needing a valid path, or clarify if it works for both files and directories, leaving usage context implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_directoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the basic action but doesn't cover critical aspects like permissions needed, error handling (e.g., for invalid paths), output format, or whether it's read-only or has side effects. This is inadequate for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded with the core purpose, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the output looks like (e.g., list of filenames, metadata), error conditions, or behavioral nuances. This leaves significant gaps for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'path' parameter clearly documented. The description adds no additional semantic context beyond what the schema provides, such as path format examples or constraints. Given the high schema coverage, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('contents of a directory'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'search_files' or 'get_file_info' which might also involve directory operations, so it lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_files' or 'get_file_info'. There's no mention of prerequisites, context, or exclusions, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the basic action. It doesn't disclose important behavioral traits like error handling (e.g., what happens if the file doesn't exist), encoding assumptions, or size limitations, leaving significant gaps for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's appropriately sized for a simple tool and front-loads the core action, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is too minimal. It doesn't explain what the return value looks like (e.g., raw text, structured data) or address potential constraints, making it incomplete for effective agent use despite the simple parameter schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'path' parameter clearly documented. The description doesn't add any parameter details beyond what the schema provides, so it meets the baseline for high schema coverage without adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Read') and resource ('contents of a text file'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_file_info' which might also read file metadata, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_file_info' for metadata or 'search_files' for finding files. The description only states what it does, not when it's appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('search for files') but doesn't describe key behaviors such as whether the search is recursive, case-sensitive, or supports regex beyond wildcards, nor does it mention performance aspects like timeouts or result limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a list of file paths, metadata), error conditions, or behavioral nuances, leaving significant gaps for a search operation with two required parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters. The description adds minimal value beyond the schema by implying the search is based on filename patterns, but it doesn't provide additional context like examples of valid patterns or directory path formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('search') and resource ('files'), specifying the search criteria ('by name pattern') and scope ('in a directory'). However, it doesn't explicitly differentiate from sibling tools like 'list_directory' or 'get_file_info', which might also involve file operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_directory' (which might list all files) or 'get_file_info' (which might retrieve metadata), leaving the agent to infer usage context without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.