Server Quality Checklist

Tool Scores

• analyze_imageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Get detailed metadata' implies a read-only operation, it doesn't specify what 'detailed metadata' includes, whether there are rate limits, authentication requirements, or any side effects. For a tool with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a simple tool with one parameter and is perfectly front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for proper tool understanding. It doesn't explain what 'detailed metadata' includes (EXIF data, dimensions, format, etc.), nor does it address potential error conditions or return format. For a tool that presumably returns structured data about images, this leaves too much unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'path' clearly documented in the schema. The description doesn't add any parameter semantics beyond what the schema already provides - it doesn't clarify what format the path should take, what file systems are supported, or any constraints on the image file. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get detailed metadata') and resource ('about an image'), making the purpose immediately understandable. However, it doesn't differentiate this from what sibling tools might do - for example, 'scan_directory' might also provide metadata about images. The description is specific but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance about when to use this tool versus alternatives like 'scan_directory' (which might provide metadata about multiple files) or 'convert_format'/'resize_image' (which modify images rather than analyze them). There's no mention of prerequisites, appropriate contexts, or exclusions for this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_formatC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool converts an image format but lacks details on permissions, side effects (e.g., file overwriting), error handling, or performance aspects. For a mutation tool with zero annotation coverage, this is a significant gap, scoring a 2.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it highly concise and well-structured, earning a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (format conversion with 4 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects, usage context, or output details, leaving gaps for an AI agent to understand full tool behavior. This inadequacy scores a 2.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters (input, output, format, quality) with descriptions and an enum for format. The description adds no additional parameter semantics beyond what the schema provides, such as usage examples or constraints. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('convert') and resource ('an image'), specifying the transformation to a different format. It distinguishes from siblings like 'analyze_image' (analysis vs. conversion) and 'resize_image' (size change vs. format change), though it doesn't explicitly name these alternatives. The purpose is specific but lacks explicit sibling differentiation, warranting a 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'analyze_image' or 'resize_image'. It implies usage for format conversion but offers no context on prerequisites, exclusions, or specific scenarios. This minimal guidance is insufficient for informed tool selection, scoring a 2.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resize_imageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'resize an image and save to a new file', implying a mutation operation, but fails to detail critical aspects like file format support, permission requirements, error handling, or whether the original file is altered. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is front-loaded and clear, making it easy to grasp the purpose quickly, which aligns well with conciseness standards.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with 5 parameters and no annotations or output schema, the description is insufficient. It lacks details on behavioral traits, error conditions, output expectations, and usage context, making it incomplete for effective agent invocation in a real-world scenario.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with all parameters well-documented in the input schema. The description adds no additional meaning beyond the schema, such as explaining parameter interactions or default behaviors. However, since the schema is comprehensive, a baseline score of 3 is appropriate as the description doesn't need to compensate for gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('resize') and resource ('image') with the outcome ('save to a new file'), making the purpose evident. However, it doesn't differentiate from sibling tools like 'convert_format' or 'analyze_image', which might also involve image processing, so it's not fully specific to sibling context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'convert_format' for format changes or 'analyze_image' for analysis. It lacks any context about prerequisites, scenarios, or exclusions, leaving usage decisions unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_directoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('scan') and output ('return metadata'), but lacks details on permissions needed, error handling, rate limits, or what metadata includes (e.g., file sizes, formats). This is inadequate for a tool that interacts with the file system.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and output. It avoids unnecessary words and gets straight to the point, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of file system operations and the lack of annotations and output schema, the description is insufficient. It doesn't explain what 'metadata' entails, potential errors, or security considerations. For a tool with no structured output, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters. The description adds no additional meaning beyond the schema, such as examples or constraints on the 'directory' path. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('scan') and resource ('directory for images'), and it mentions the output ('return metadata'). However, it doesn't explicitly differentiate from sibling tools like 'analyze_image' or 'convert_format', which might also involve image processing but with different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios where scanning for images is preferred over other operations or prerequisites like file system access. This leaves the agent without context for tool selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.