Server Quality Checklist

Tool Scores

• describe_sceneB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description alone must disclose behavior. It reveals that the output will include spatial relationships, atmosphere, and context, but omits any details about output format, potential limits, errors, or differences from analyzing generic images. This is insufficient for a tool with no structured behavioral cues.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and key outputs without redundant filler. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple with two parameters and no output schema. The description provides a reasonable overview of what the description will contain, but lacks guidance on result format or examples. Given the existence of overlapping siblings, more context on when exactly to use this tool would be valuable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides full descriptions for both parameters (image and focus), so the tool description adds no new parameter semantics. Baseline 3 applies because schema coverage is 100%, and the description's mention of scene context loosely complements the 'focus' parameter but adds no technical detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get a detailed description of a scene' and specifies key content like spatial relationships, atmosphere, and context. This distinguishes it from generic image analysis and text extraction, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a scene-level description is needed, but gives no explicit guidance on when to prefer this over siblings like analyze_image. No exclusions or alternative conditions are mentioned, so usage context is only moderately clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_imageA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions using 'vision AI' and lists output types, but does not disclose whether the operation is read-only, whether data is stored, potential latency, network requirements, or side effects. This is a significant gap for safe tool invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loaded with the core purpose. It efficiently conveys input format and output summary without redundancy or filler. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters and no output schema, the description adequately summarizes what the tool does and the general nature of its returns. However, it lacks details about the response structure or behavior under different detail levels, which would be helpful given the absence of an output schema. Still, it is sufficiently complete for a general analysis tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter (image, detail, prompt) fully described in the schema. The description's mention of 'Provide either a URL or base64-encoded image data' simply restates the schema's image parameter description. It adds no new meaning beyond what the schema already provides, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Analyze') and resource ('an image'), and lists concrete output categories (objects, text, colors, scene description) that clearly distinguish it from sibling tools like compare_images, extract_text, and describe_scene. This makes the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies general-purpose analysis by listing multiple output types, but it does not explicitly state when to use this tool vs. alternatives like extract_text or describe_scene. There is no mention of exclusions or when-not-to-use scenarios, only input format guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_imagesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states that the tool produces a description of differences, similarities, or relationships, which conveys the core output behavior. However, it does not disclose potential limitations, failure modes, or the exact format of the output, leaving behavioral details ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately states the action and resource. It avoids unnecessary detail and is well-structured, earning a high score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, but with no output schema, the description should clarify the return structure. It states the tool will 'describe' differences, but not whether the output is a single combined summary or per-pair comparisons. This ambiguity makes the description only minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the schema (100% coverage), so the baseline is 3. The description doesn't add further semantic meaning beyond the schema, though it does reinforce the 'two or more images' constraint. No additional parameter context is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function—comparing multiple images and describing their differences, similarities, or relationships—and distinguishes it from siblings like analyze_image and describe_scene, which focus on single images. The verb 'compare' and the explicit 'two or more images' scope make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for comparing multiple images but does not explicitly mention when to prefer this tool over siblings such as analyze_image or describe_scene. There are no explicit alternatives or exclusions, so guidance is only implicit through the requirement of at least two images.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_textA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full responsibility for behavioral disclosure. It states the OCR action but does not describe the output format, potential limitations (e.g., handwriting, low-quality images), or error behavior. This is a significant gap given the complete lack of annotation context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the core action and adds a brief, relevant list of use cases. Every part is purposeful, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with a well-described schema, the description offers enough information to select and invoke it correctly. The absence of an output schema means the return value is not explicitly stated, but 'Extract and transcribe text' reasonably implies text output; a brief note on return format would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides thorough descriptions for both parameters (image accepts URL or base64; preserve_formatting defaults to true). Schema description coverage is 100%, so the description adds no additional parameter-level meaning, which matches the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb phrase 'Extract and transcribe text from an image' and explicitly identifies the operation as OCR. This clearly distinguishes it from sibling tools like analyze_image, compare_images, and describe_scene, which focus on image understanding rather than text extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage contexts ('documents, screenshots, signs, etc.') that help an agent decide when to apply this tool. It does not explicitly state when not to use it or mention alternatives, so it falls short of full exclusion guidance but offers more than implied usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.