Server Quality Checklist

Tool Scores

• generate_imageB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations exist but are minimal (readOnlyHint=false, etc.). Description adds value by explaining auto-selection behavior and provider strengths. However, it does not disclose important behaviors like file saving (implied by output_path), potential latency, or cost implications. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with headings, lists, and examples. It is front-loaded with the main purpose. However, it is somewhat lengthy with provider comparisons that could be condensed. Overall, sentences earn their place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (multiple providers, many parameters, auto-selection), the description explains the selection logic and provides examples. It mentions override and return format. An output schema exists to handle return details. It is fairly complete for an agent to understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the top-level param 'params' is 0% (no description), and the description only repeats 'Image generation parameters including prompt and optional settings' – adding no new meaning. While nested properties have descriptions in the schema, the description fails to compensate for the top-level lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates an image using the best available provider, with specific verb 'Generate' and resource 'image'. It distinguishes from siblings by focusing on single image generation with auto-selection, while siblings like 'edit_image' and 'conversational_image' imply different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides detailed guidance on when to use each provider within the tool, but offers no guidance on when to choose this tool over sibling tools like 'edit_image' or 'generate_image_batch'. An agent would need to infer from the tool name and purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conversationsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read and idempotent. The description adds that results are 'recent' and can be 'continued for refinement', which provides some behavioral context beyond the annotations. However, it doesn't detail behavior like empty results, ordering, or response structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: four sentences total, with the core purpose in the first sentence. No extraneous information. All sentences add value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool, the description covers purpose and key traits (recent, continuable, tracks provider/history). It lacks details on pagination behavior and ordering, and the output format is only implied via the parameter. Still, it is mostly sufficient given the annotations and schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description only says 'params: Options for filtering and formatting the list.' This is vague and does not add meaning beyond the input schema, which already describes each parameter. With 0% schema description coverage, the description fails to compensate, but the schema itself is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists saved image generation conversations, using specific verb 'list' and resource 'conversations'. It distinguishes from siblings as no other list tool exists. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to retrieve recent conversations for refinement, but provides no explicit guidance on when to use it versus other tools or when not to use it. No alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• conversational_imageA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=false (mutation) and destructiveHint=false. The description adds behavioral traits beyond annotations: provider locking across conversations (cannot switch mid-conversation), dialogue modes impact on interaction depth, and the usage pattern for continuation via conversation_id. These details are useful but not exhaustive (e.g., no mention of output_path creation).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, when-to-use, dialogue modes, provider selection, usage pattern, returns). It is concise and front-loaded with essential information. The usage pattern could be slightly shorter, but overall it efficiently conveys the tool's workflow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (conversational image generation with many parameters and multi-turn refinement), the description covers core concepts, when to use, dialogue modes, provider locking, and continuation pattern. The output schema exists, so return details are not needed. It lacks discussion of some advanced parameters (e.g., reference_images, input_image_file_id), but schema handles those. It is sufficient for an AI agent to select and invoke.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (main description does not detail individual parameters), but the schema itself has thorough descriptions for all parameters. The description adds some context for 'dialogue_mode' (enum values) but largely repeats schema info. Baseline is 3 due to high schema coverage, and no significant extra meaning is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate images conversationally with iterative refinement.' It specifies the resource (images) and the verb (generate with conversation), and distinguishes from sibling tools like 'generate_image' by emphasizing conversation and iterative refinement. The dialogue modes further clarify the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists 'USE THIS TOOL when:' conditions, such as vague prompts, iterative refinement, or user asking for guidance. It does not directly state when not to use it (e.g., when prompt is clear and no refinement needed), but the context effectively implies when alternatives like 'generate_image' are better. It also covers dialogue modes and provider selection, offering solid guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_gemini_modelsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint. The description adds that it queries the Gemini API and returns capabilities, but does not provide additional behavioral details beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with a clear purpose, action, return description, and use case. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero parameters and presence of an output schema, the description fully covers the tool's purpose and typical use case without needing further elaboration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the baseline is 4. The description does not need to add parameter semantics, and it does not attempt to.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists available Gemini models that support image generation. This verb-noun pair is specific and distinguishes it from siblings like 'generate_image' or 'list_providers'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions it is useful for troubleshooting or choosing alternative models, providing clear context. It does not explicitly state when not to use it, but the context of sibling tools offers implied guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_costA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint and idempotentHint. The description adds behavioral details: it runs the same provider auto-selection as generate_image, uses a local pricing table, and notes the estimate is a ballpark. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: starts with purpose, then explanation, caveats, and finally Args/Returns. It is slightly verbose but front-loaded with key information. A bit more conciseness could improve it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (cost estimation with no side effects), the description covers all essential aspects: what it does, how it works, caveats, and return format (via output schema). With annotations and output schema present, no gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description's Args section briefly lists parameters but adds little new meaning beyond the schema's own descriptions. The schema already provides detailed descriptions for each parameter. Schema coverage is 0% by context definition, but the schema itself is informative, so the description does not compensate for missing schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: estimating the cost of generating an image without actually generating it. It uses specific verb 'estimate' and resource 'cost', and distinguishes it from siblings like generate_image by explicitly noting it does not generate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use: before generating, for comparing providers/qualities. It implies not to use for actual generation, but does not explicitly state when not to use or list alternatives beyond the tool itself. However, the sibling list provides context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_providersA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, but the description adds value by detailing the return content: API key configuration, best use cases, and feature comparison. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a clear lead sentence and bullet points for output details. Every sentence adds value, and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately explains the tool's purpose and what the output includes, even without needing to detail parameters. Given the presence of an output schema, the description is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema coverage is 100%. With zero parameters, the baseline is 4, and the description does not need to add parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'List available image generation providers and their capabilities', clearly identifying the verb (list) and resource (providers with capabilities). This distinguishes it from sibling tools like generate_image or edit_image, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises 'Use this to understand which provider to choose for your task', providing clear context for when to use this tool. While it doesn't list explicit alternatives or when-not-to-use, the context of sibling tools makes the usage well implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_image_batchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate non-readonly, non-destructive, non-idempotent, open-world. Description adds concurrency bounds, failure isolation, rate limit dependency, and result format, providing substantial behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with summary, args, and returns. Every sentence adds value; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers inputs, behavior, concurrency, error handling, and return format. With output schema present, it provides complete guidance for a batch image generation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Top-level 'params' description is brief, but the nested schema for `BatchGenerationInput` has detailed field descriptions. Tool description adds context about batch structure and default provider, complementing the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool generates many images concurrently from a list of prompts. Distinguishes from siblings like `generate_image` by mentioning batch processing and concurrency.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this tool instead of calling `generate_image` in a loop, with a concrete performance example. Also describes per-item failure isolation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_imageA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behavioral traits: uses /images/edits endpoint, default high input fidelity to preserve pixels, mask support, sequential workflow. Annotations (readOnlyHint=false, destructiveHint=false) are consistent; description adds context about mutable but non-destructive behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with bullet points and sections (key features, typical workflow). No fluff; each sentence adds valuable information. Appropriate length given tool complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers use cases, workflow, and key features comprehensively. With an output schema present, return value explanation is unnecessary. The description is self-contained for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the tool has a single 'params' object with 0% description coverage at top level, the nested schema thoroughly documents all parameters. The description adds value by explaining key parameters like input_fidelity and mask_path in context, aiding interpretation beyond schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it edits existing images using OpenAI's gpt-image-2 endpoint. It lists specific use cases (image-to-image refinement, inpainting, sequential edits) that distinguish it from sibling tools like generate_image or conversational_image, which focus on generation or conversation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance under 'This is the right tool for:' with bullet points covering refinement, inpainting, sequential edits, and brand modifications. It implies not for from-scratch generation (handled by generate_image). The workflow description further clarifies when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.