Server Quality Checklist

Tool Scores

• analyze_imageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist. The description mentions support for local paths and URLs but omits details about output format, file size limits, or side effects. It does not specify that the tool returns a text description or answer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with essential information, no redundancy. However, could be expanded to include key constraints without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no output schema, and no annotations, the description should explain the return format and limitations. It does not specify what the tool returns (e.g., a text description) or any constraints like file format support.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters have schema descriptions (100% coverage). The description adds context for the 'image' parameter by noting local path and URL support, but does not enhance detail or prompt beyond their schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (Analyze) and resource (image), and specifies support for local file paths and URLs. It differentiates from sibling tools like analyze_video and ocr_image by mentioning vision language model, but does not explicitly contrast with compare_images.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., compare_images, ocr_image). No exclusions or prerequisites provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ocr_imageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description should disclose behavioral traits. It only states it extracts text via OCR but does not mention read-only nature, error handling, performance, or authentication needs. The description adds no behavioral context beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences that immediately convey the core purpose and output options. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters and no output schema or annotations, the description lacks completeness. It does not explain potential failures, image format support, or how to interpret results. The missing context makes it less useful for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are covered in the input schema (100% coverage). The description adds minimal extra meaning by naming output formats, but this aligns with the format enum. No significant semantic enhancement beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action (extract text using OCR) and specifies supported output formats. It is specific enough to distinguish from sibling tools like analyze_image or compare_images, though it does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance on when to use this tool. It lists output formats but does not compare against sibling tools or specify conditions for use (e.g., image quality, file size limits). There is no when-not-to-use advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_videoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only mentions model requirements, omitting details like processing speed, output format, potential errors (e.g., unsupported video formats), or whether videos are processed entirely. The agent lacks critical behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the primary action and then a key requirement. Every word is purposeful; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (two parameters, no output schema), the description is adequate but could be improved by stating what the output is (e.g., returns text) and any limitations (e.g., video length). It leaves some context gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema describes both parameters (video and prompt) with clear documentation, covering 100% of properties. The description does not add additional semantics beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes video content using a vision language model. It implicitly distinguishes from sibling tools like analyze_image (images) and OCR (text in images) by specifying video support. However, it lacks explicit mention of the analysis type beyond general AI interpretation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes the requirement for a model with video support, implying conditions for use, but gives no explicit guidance on when to use this tool versus siblings (e.g., vs analyze_image for static frames). The context of sibling names provides some implicit differentiation, but the description does not state when-not-to-use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_imagesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It fails to disclose any behavioral traits such as return format, side effects, auth needs, or limitations. For a tool with no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is two concise sentences with no wasted words. It is front-loaded with the main action and covers the key detail about supported input types.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (2 params, no output schema, no annotations), the description is adequate but incomplete. It does not mention return format or potential limitations like unsupported image formats, which would be helpful for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%; both parameters have descriptions. The description adds no additional meaning beyond what the schema provides. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'compare 2-4 images and describe differences/similarities', which is a specific verb+resource combination. This distinguishes it from sibling tools like analyze_image (single image analysis) and ocr_image (text extraction).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for comparing images and supports file paths and URLs, but it does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives. Usage context is implied, not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.