Server Quality Checklist

Tool Scores

• text_to_imageC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior, but it only mentions return values (task id, status, output URLs). It omits important details like potential blocking behavior (wait parameter), polling mechanics, asynchronous task lifecycle, and any rate limits or auth requirements. This is insufficient transparency for a task-creation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the core purpose and returns. It contains no redundancy or filler, making it appropriately concise for a tool whose complexity is not conveyed elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity—8 parameters, asynchronous behavior, no annotations, and no output schema—the description is drastically under-specified. It lacks usage guidance, parameter semantics, and behavioral disclosures beyond a minimal return statement, rendering it incomplete for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25%, and the tool description does not explain any of the 8 parameters. With 3 required parameters (prompt, aspect_ratio, quality) and additional control parameters (wait, timeout_ms, callback_url, poll_interval_ms), the lack of parameter meaning in both schema and description makes it impossible for an agent to correctly set values without external knowledge.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Create a GPT Image task') and resource (RunAPI, text to image), which distinguishes it from sibling tools like edit_image and get_task. It is not explicitly contrasted with those tools, but the creation/generation purpose is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It implies usage for generating text-to-image tasks but does not mention exclusions, prerequisites, or comparisons to edit_image or get_task, leaving the agent without clear selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only states the basic purpose. It does not indicate whether the tool is read-only, what data is returned, if authentication is required, or any side effects. The agent lacks critical behavioral information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that conveys the core function with no extraneous words. It is front-loaded and efficient, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with optional parameters and no output schema, the description is adequate but not complete. It omits details about what the returned pricing information includes (e.g., per-token costs, endpoint-specific rates). The description could be improved by mentioning the output format or the scope of pricing data provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters with enums and descriptions. The description adds minimal value beyond confirming the model line scope. The default behavior (primary model, default endpoint) is implied but not explicitly stated, and there is no elaboration on how parameters affect the result.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up pricing for the gpt-image model line. It identifies the resource (pricing) and action (look up), and the sibling tools (edit_image, text_to_image) are clearly different in purpose, so no confusion. However, 'Look up' is somewhat generic and could be more specific like 'Retrieve pricing information'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs alternatives. It does not mention prerequisites, when to omit parameters, or when to use sibling tools for specific actions. The agent is left to infer usage without explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_imageB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions returning a task id and status, hinting at asynchronous behavior, but does not explain polling, wait semantics, callback_url, or how output URLs become available. It omits authentication requirements, rate limits, and whether the operation is blocking. The description provides minimal insight into the task lifecycle.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the action ('Create a GPT Image task') and includes essential context (edit image, return values). There is no redundancy or wasted words, exemplifying efficient structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters, no output schema, and no annotations, the description is too sparse to be complete. It does not explain the overall workflow (e.g., pairing with get_task to retrieve results), the async behavior controlled by wait, or how callback_url and poll_interval_ms function. The description leaves significant gaps for a complex task-based tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 22% (2 of 9 parameters have descriptions). The tool description adds no parameter details beyond the name 'edit image,' failing to clarify required parameters like prompt, source_image_urls, aspect_ratio, or quality. It does not compensate for the low schema coverage, leaving the agent without meaningful guidance on how to construct valid requests.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Create a GPT Image task on RunAPI (edit image).' This distinguishes it from sibling tools like text_to_image by explicitly indicating it handles image editing. It also states return values (task id, status, output URLs), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for image editing but provides no explicit guidance on when to use this tool versus alternatives like text_to_image. It does not mention prerequisites (e.g., needing a source image URL) or complementary tools like get_task for retrieval. Guidance is only implicit through the tool's name and the phrase 'edit image.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the non-mutating nature via the verb 'fetch' and scopes to gpt-image tasks, but does not mention expected behaviors like polling intervals, error handling, or that the task might still be processing. This is acceptable but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and free of unnecessary wording. It front-loads the core purpose and earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch tool with no output schema and no annotations, the description conveys the primary function but omits details about what the status values might be or whether the result payload is always present. This is a minor gap given the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with descriptions for task_id and action. The description does not add meaning beyond the schema, but the schema already provides clear semantics for both parameters, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the current status and latest result payload for a gpt-image task. The verb 'fetch' and specific resource ('status and latest result payload') distinguish it from sibling creation tools like edit_image and text_to_image.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to poll an asynchronous gpt-image task for its status/result, but it does not explicitly state when to use it versus alternatives or provide exclusions. The context of siblings suggests it is the retrieval counterpart to the creation endpoints, but this is not made explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits: it opens a browser (interactive), performs a PKCE flow, and writes to a config file. This goes beyond what annotations (none) provide. However, it does not mention potential issues like browser dependency or timeouts, so it is not a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that conveys all essential information without any unnecessary words. It is concise and front-loaded with the action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description does not explain what the tool returns (e.g., success confirmation or API key). It focuses on the side effect (saving to config) but omits the immediate output. For a login tool, this is a notable gap, though the description is otherwise comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters, with the 'force' parameter already described in the schema. The description does not add additional meaning about the parameter beyond what the schema provides; it is separate. Thus a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: authenticating RunAPI via a browser PKCE login flow and saving the API key to a config file. It uses specific verbs ('Authenticate', 'opening', 'saving') and identifies the resource ('RunAPI') and the output location. This distinguishes it from sibling tools which deal with pricing, image editing, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use the tool or when to avoid it. It implies it is for initial authentication, but lacks guidance on scenarios such as already having a valid key or needing to re-authenticate. No mention of alternatives or prerequisites is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.