Server Quality Checklist

Tool Scores

• list_directoryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the root path configuration and recursive behavior, but it doesn't cover important aspects like whether this is a read-only operation, potential rate limits, error conditions (e.g., invalid paths), or what the output format looks like (since there's no output schema). For a tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise and front-loaded: it states the core purpose in the first sentence, followed by key contextual details. Both sentences earn their place by providing essential information without redundancy. It's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description is incomplete. It covers basic purpose and some parameter hints but lacks crucial behavioral details (e.g., safety, errors, output format) and doesn't fully compensate for the missing structured data. For a 4-parameter tool with no annotations or output schema, this description should do more to guide the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema: it mentions the root path context and hints at the 'recursive' parameter's effect. However, it doesn't provide additional semantic context for parameters like 'max_depth' or 'max_entries' that aren't covered in the description. Baseline 3 is appropriate when the schema does most of the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List files and directories.' It specifies the verb ('List') and resource ('files and directories'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'fetch_url' or 'read_file', though the distinction is somewhat implied by the domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: 'Paths are relative to the configured root (/app/workspace). Set recursive=true to list all nested files.' This gives basic guidance on when to use certain parameters, but it doesn't explicitly state when to use this tool versus alternatives like 'fetch_url' or 'read_file', nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transform_dataA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions what the tool does, it doesn't describe important behavioral aspects like error handling, performance characteristics, rate limits, authentication requirements, or what happens with malformed input. The description is functional but lacks operational transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence states the core functionality with specific format examples, and the second sentence provides usage contexts. No wasted words, front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no annotations and no output schema, the description is adequate but has clear gaps. It explains what the tool does and when to use it, but doesn't address output format details, error conditions, or behavioral constraints. Given the complexity and lack of structured metadata, more complete operational guidance would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 75% schema description coverage, the baseline is 3. The description doesn't add specific parameter semantics beyond what's in the schema - it mentions format conversions generally but doesn't explain parameter interactions, constraints, or edge cases. The schema already documents parameters well, so the description doesn't compensate for the 25% coverage gap but doesn't need to either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('convert', 'reformat', 'prepare', 'normalise') and resources ('data between formats'), listing all supported formats. It distinguishes this from sibling tools (fetch_url, list_directory, read_file) by focusing on data transformation rather than data retrieval or file operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('useful for reformatting API responses, preparing data for display, or normalising spreadsheet exports'), giving concrete scenarios. However, it doesn't explicitly state when NOT to use it or mention alternatives among sibling tools, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_urlA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: it returns response body, status code, and content type; rejects binary content; and supports specific protocols. However, it misses details like error handling, rate limits, or authentication needs, which would be useful for a fetch operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and efficiently adds critical details in two sentences. Every sentence earns its place by specifying functionality, protocols, return values, and content restrictions without redundancy, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is mostly complete—it covers purpose, behavior, and limitations. However, it lacks details on error responses or output structure, which would enhance completeness for an agent invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (2 out of 3 parameters have descriptions). The description adds no specific parameter semantics beyond what the schema provides—it mentions URL fetching generally but does not explain headers or timeout usage. With moderate schema coverage, the baseline score of 3 is appropriate as the description does not compensate for gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('fetch the content of a URL') and resource ('URL'), distinguishing it from sibling tools like list_directory, read_file, and transform_data. It specifies the return format ('as text') and protocol support ('HTTP and HTTPS'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning protocol support and content restrictions ('text and JSON only'), but it does not explicitly state when to use this tool versus alternatives or provide context about prerequisites. It lacks direct guidance on scenarios where this tool is preferred over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing important behavioral traits: path traversal blocking, root directory context, and encoding recommendations for binary files. It doesn't mention error conditions, permissions, or rate limits, but provides solid operational context for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with zero waste - each sentence provides essential information: core purpose, path constraints, and encoding guidance. Perfectly front-loaded with the main action first, followed by important operational details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read operation with no annotations and no output schema, the description provides good context about path handling and encoding. It could mention what happens with non-existent files or permission errors, but covers the essential operational constraints well given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema - it mentions encoding=base64 for binary files (which is also in the schema) and implies path handling context. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Read a file') and resource ('from the filesystem'), distinguishing it from sibling tools like list_directory (which lists files) or fetch_url (which retrieves from URLs). It provides specific context about path handling that makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use certain options ('use encoding=base64 for binary files') and mentions path traversal restrictions, but doesn't explicitly contrast when to use this tool versus alternatives like fetch_url or transform_data. It gives operational guidance but not sibling differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.