Server Quality Checklist

Tool Scores

• edit_imageB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of disclosure. It mentions returning a task id, status, and output URLs, which hints at asynchronous processing, but it does not explain that output URLs may only be available after task completion, how to poll, or whether the operation has side effects (e.g., creating a new image). The safety checker, callback, and polling parameters are not explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loaded with the verb and resource. It contains no unnecessary words, but it sacrifices critical detail for brevity. Every sentence contributes, but the overall structure could be more informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, no annotations, and no output schema, the description is severely insufficient. It lacks guidance on async workflows, parameter values, and edge cases. The existence of sibling tools like get_task suggests a workflow that is not described here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 18% (model and wait only). The description provides no parameter-specific meaning. The phrase 'edit image' implies the existence of a source image and prompt, but it does not name or explain the required parameters (prompt, source_image_url) or optional behaviors. The description does not compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Create') and resource ('Qwen 2 task on RunAPI') with the parenthetical '(edit image)' distinguishing it from the sibling text_to_image tool. It also specifies the return value structure, so the agent knows what to expect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives like text_to_image. The name and '(edit image)' imply it is for image-editing tasks, but the description does not state that this tool should be used for editing and text_to_image for generation. No exclusions or preferences are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of disclosing side effects and behavior. It states the read-only action ('Fetch'), but does not mention whether the call is safe to repeat, what happens if the task is incomplete, auth requirements, or error semantics. This leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant information. Every word contributes to defining the tool's purpose and scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The absence of an output schema and annotations means the description must explain return behavior and contextual details. It mentions 'status and latest result payload' but does not describe how status is represented, when the payload becomes available, pagination (if any), or error conditions. For a tool with only 2 parameters and no rich schema, this is a notable gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides complete descriptions for both parameters (task_id and action), covering 100% of the query parameters. The description adds no additional meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and names the resource ('qwen-2 task') and the data returned ('current status and latest result payload'). It clearly differentiates from sibling tools like login, edit_image, and text_to_image, though without explicitly naming them as alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to check on an existing asynchronous task created via siblings, but it does not explicitly state when to use it, what prerequisites exist (e.g., task_id from a prior call), or which alternatives to consider. The wording 'current status' hints at polling, but no direct guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• text_to_imageB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return shape (task id, status, output URLs) and the creating action, but does not explain the asynchronous nature, wait/polling behavior, callback mechanism, or safety checker. Behavior beyond the obvious create action is under-specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, with the main action front-loaded and the return info added second. Every sentence earns its place; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 10 parameters, no annotations, and no output schema, this description is far too sparse. It lacks guidance on configuration (aspect ratio, output format), async behavior, and error cases. The return statement is helpful but insufficient for a complex task creation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20% (2 of 10 params have descriptions). The tool description adds no parameter-level explanations – it only mentions the overall 'text to image' function. It does not clarify prompt, seed, aspect_ratio, output_format, or other meaningful parameters, which the low schema coverage does not compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Create a Qwen 2 task on RunAPI (text to image)' – a specific verb and resource. It clearly differentiates from sibling tools like edit_image and get_task by specifying the text-to-image generation use case and the RunAPI platform.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for generating images from text ('text to image') but provides no explicit guidance on when to choose it over alternatives like edit_image. No exclusions or alternative tool references are given; usage context is inferred rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the authentication method (PKCE, browser), and file path. However, it omits side effects like user interaction requirements or failure scenarios.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, efficient and front-loaded with the core verb 'Authenticate'. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the tool's behavior given its simplicity and lack of output schema. It explains what happens (saves API key to config file), though it could mention the return value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter. The description adds no additional meaning about the 'force' parameter beyond the schema's description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: authenticate by opening a browser PKCE login flow and saving the API key. It distinguishes well from sibling tools which are unrelated (image editing, pricing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance provided. The description does not specify when to use this tool (e.g., first-time setup vs. re-authentication) nor what alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'Look up' indicates a read-only operation, which is a useful behavioral signal, but it does not disclose the return format, error handling, or any other implications. It is minimally sufficient but leaves room for improvement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that directly states the purpose. It contains no redundancy or irrelevant information, and it is front-loaded with the action and subject.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with optional parameters and no output schema, the description adequately conveys the core function. However, it could be more complete by indicating what kind of pricing information is returned (e.g., per-call cost, rate limits) or noting that it is read-only.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage of parameters with descriptions and enums, so this dimension is at baseline. The description adds no extra meaning beyond what is already in the schema, nor does it clarify the relationship between 'model line' and the specific model values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Look up') and resource ('RunAPI pricing for the qwen-2 model line'). It is easily distinguishable from sibling tools like login, edit_image, and get_task, which serve entirely different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for looking up pricing but provides no explicit guidance on when to use it vs. alternatives. It lacks any mention of prerequisites, exclusions, or helpful context about the model line or parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.