Server Quality Checklist

Tool Scores

• edit_imageC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. It mentions the return of a task ID, status, and output URLs, but fails to cover important aspects such as the asynchronous nature of task creation, the need for polling (evident from parameters like 'wait' and 'poll_interval_ms'), authentication requirements, or rate limits. The description is insufficient for an agent to anticipate the tool's full behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, succinct sentence that communicates the core action and return values without extraneous detail. It front-loads the purpose ('Create a GPT Image 2 task') and efficiently includes the parenthetical clarification. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no output schema, no annotations), the description is woefully incomplete. It does not address the editing workflow, the meaning of output fields, or the interplay between parameters. The agent lacks critical context to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 8 parameters, with only 2 having descriptions (25% coverage). The tool description adds no parameter-level information, failing to compensate for the low schema coverage. The description does not explain the role of key parameters like 'prompt', 'source_image_urls', or 'aspect_ratio' in the editing context, leaving the agent without necessary guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool creates a GPT Image 2 task for editing images, using the verb 'Create' and the resource 'GPT Image 2 task'. However, it does not explicitly differentiate from the sibling tool 'text_to_image', which likely generates images. The parenthetical 'edit image' hints at the distinction but lacks explicit comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'text_to_image' or 'get_task'. It omits any prerequisites, context, or conditions for appropriate usage, leaving the agent without decision-support information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• text_to_imageB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It indicates the tool creates an asynchronous task (via the return of a task id) and supports polling via wait/timeout parameters. However, it does not explicitly state the asynchronous nature, authorization requirements, or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that covers the core purpose and outputs without unnecessary verbosity. It could be slightly more structured but is effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and no output schema or annotations, the description is minimal. It fails to explain the asynchronous workflow, output format details, or parameter dependencies, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 29% schema description coverage, the description adds no additional parameter information. Parameters like aspect_ratio and output_resolution are left unexplained, and the description does not compensate for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create a GPT Image 2 task'), the resource ('on RunAPI'), and the expected outputs ('Returns a task id, status, and output URLs'). It is distinct from sibling tools like edit_image and get_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as edit_image or check_pricing. The description does not specify prerequisites or scenarios where this tool is appropriate or inappropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It correctly characterizes it as a lookup, implying read-only, but does not mention permissions, rate limits, or whether results are cached. The minimal description is adequate but does not exceed expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that directly states the tool's function with no unnecessary words or repetitions. It is optimally concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple pricing lookup with optional parameters, the description captures the core purpose. However, it does not mention that parameters are optional or that defaults apply, which could leave an agent unsure about what happens if no arguments are provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with parameter descriptions already defining defaults and enums. The tool description adds no new meaning about how parameters affect pricing results, so it meets the baseline but provides no extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('look up') and resource ('pricing'), and identifies the exact model line ('gpt-image-2'). It clearly distinguishes from sibling tools like edit_image or text_to_image, which perform actions rather than lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the sibling names imply this is for pricing queries rather than image editing, the description provides no explicit guidance on when to use this tool versus alternatives, nor any conditions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full transparency burden. It indicates a read-only operation ('Fetch'), which is appropriate, but it does not disclose any behavioral traits such as error handling, rate limits, or authorization requirements. The description is minimal and lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence of 13 words. It is front-loaded with the action and resource, making it easy to process quickly. Every word is meaningful with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch tool with no output schema, the description adequately states what is returned (status and latest result payload). However, it does not explain the format or potential error conditions. Given the tool's simplicity, it is mostly complete but could be slightly more detailed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters (task_id and action) are fully described in the input schema (100% coverage). The description does not add additional meaning beyond what the schema provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch' and the specific resource 'current status and latest result payload for a gpt-image-2 task'. It effectively distinguishes this tool from siblings like check_pricing, edit_image, and text_to_image by specifying its role as a status retrieval tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a task has been created and its status is needed, but it lacks explicit guidance on when not to use it or mention of alternative tools. No exclusions or prerequisites are provided, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the browser flow and config file saving, offering good transparency. However, it could detail side effects like overwriting existing credentials or the behavior if already authenticated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no extraneous words, effectively communicating the tool's action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple authentication tool, the description is fairly complete, covering action and effect. It could mention return values or that subsequent calls are authenticated, but this is not a major gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds no value beyond the schema's description of the 'force' parameter. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Authenticate' and the resource 'RunAPI', specifying the PKCE login flow and file path. It uniquely distinguishes from sibling tools like check_pricing and edit_image, which handle unrelated tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for authentication but does not provide explicit guidance on when to use this tool versus alternatives, nor does it mention prerequisites or when to avoid it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.