Server Quality Checklist

Tool Scores

• text_to_imageC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits, but it only states that the tool creates a task and returns output. It does not mention asynchronous behavior, authentication needs, rate limits, costs, or any side effects, leaving significant ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded, but it omits crucial details. It could be slightly longer to include parameter hints or usage context without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters, no output schema, and no annotations, the description is under-specified. It does not explain the task lifecycle, how 'wait' and 'poll_interval_ms' interact, or the meaning of output URLs, making it inadequate for robust tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 22%, yet the description adds no parameter explanations beyond the schema's minimal descriptions for 'wait' and 'model'. Parameters like 'prompt', 'seed', 'aspect_ratio' remain undocumented in both schema and description, failing to compensate for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates an Imagen 4 task for text-to-image generation and returns task id, status, and output URLs. The verb 'Create' and resource 'Imagen 4 task' are specific, and the parenthetical '(text to image)' distinguishes it from sibling tool 'remix_image'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like 'remix_image' or 'get_task'. The description does not mention prerequisites, conditions, or context for selection, leaving the agent to infer usage solely from the purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remix_imageB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose that the tool creates an asynchronous task and returns a task id, status, and output URLs. However, it does not mention polling behavior, authentication, or failure scenarios, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and front-loaded, with no extraneous information. Every word contributes to the core purpose and return values.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (10 parameters, no output schema, no annotations), the description is too sparse. It covers the basic purpose and return shape but omits crucial context like parameter meanings, async behavior, and usage scenarios, making it inadequate for an AI agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain any of the 10 parameters. Schema description coverage is only 20%, and the description fails to compensate, leaving parameters like prompt, aspect_ratio, and output_resolution without contextual meaning beyond their raw schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool creates an Imagen 4 task for remixing images and indicates the return values (task id, status, output URLs). The phrase 'remix image' distinguishes it from the sibling text_to_image tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives like text_to_image or get_task. The description only states the function without contextual usage tips or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. 'Look up' implies read-only, but no explicit disclosure of safety, authentication needs, or rate limits. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 9 words, front-loaded with verb and resource. No waste. Highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema; description does not hint at return format or pricing details. As a simple lookup, missing some context about output structure, but adequate for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions and enums for both parameters. Description adds only that the tool looks up pricing for the model line, not much beyond schema. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Look up', resource 'pricing', and scope 'for the imagen-4 model line'. Distinguishes from siblings (get_task, remix_image, text_to_image) which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. No prerequisites, exclusions, or context provided about selecting this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the core behavior (browser login, PKCE, file saving) but does not mention side effects like overwriting existing config or what happens if already authenticated. This lack of detail affects transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the key action. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is fairly complete for a simple login tool with one parameter. It covers the authentication flow and file saving but omits details like success indication or behavior when already authenticated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for the 'force' parameter. The tool description adds no extra information beyond the schema, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool authenticates RunAPI via a browser PKCE login flow and saves the API key to a specific file. This distinguishes it from sibling tools like check_pricing or text_to_image.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for authentication before using other RunAPI tools. The 'force' parameter provides guidance on when to re-run login. However, it does not explicitly state when not to use it or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Fetch' clearly indicates a read-only operation, and the description states what data is retrieved (status and result payload). It does not mention potential errors or polling behavior, but for a simple fetch tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately communicates the tool's function without any redundant or extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch operation with fully documented parameters, the description sufficiently covers the tool's purpose and output (status and payload). The absence of an output schema is mitigated by the explicit mention of what is returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents both parameters with descriptions (task_id, action) and an enum for action. The tool description adds no additional parameter-specific meaning beyond indicating the task context, so the baseline of 3 applies due to high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Fetch' and identifies the resource as 'current status and latest result payload for an imagen-4 task,' which precisely matches the tool name and distinguishes it from sibling generation tools like remix_image and text_to_image.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is used to retrieve the status/result of a previously created imagen-4 task, and the parameters (task_id, action) reinforce that. It does not explicitly state 'use this after calling text_to_image or remix_image,' but the context is clear from the tool name and sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.