Server Quality Checklist

Tool Scores

• text_to_imageB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral burden. It discloses that the tool returns a task id, status, and output URLs, but it does not mention the asynchronous task nature, polling behavior, safety checker, or prerequisite login/authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary action ('Create a Flux 2 task'). It efficiently conveys the return value but could be slightly more structured with an additional context sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 10 parameters and no output schema, yet the description only covers the basic creation and return values. Important context such as model options, aspect ratios, safety checking, and async completion is missing, making it insufficient for a complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80%, so the schema already documents most parameters. The description adds no parameter-level details beyond what the schema provides, so it neither elevates nor detracts from the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Create a Flux 2 task on RunAPI (text to image).' It clearly states the tool's function and differentiates from siblings like remix_image or get_task by focusing on text-to-image generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for text-to-image generation but does not explicitly state when to use this tool versus alternatives such as remix_image. It lacks exclusions or guidance on when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It merely states 'look up' without mentioning read-only nature, required authentication, output format, or any side effects, leaving significant ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the verb and resource, with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 optional params, no output schema), the one-line description is minimally viable but incomplete: it does not explain what the return value contains or the effect of default parameter values, leaving gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents both parameters (model and action) with enums and descriptions (100% coverage). The description adds no parameter-specific meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('look up') and identifies the exact resource ('RunAPI pricing for the flux-2 model line'), clearly distinguishing it from sibling tools like 'remix_image' and 'text_to_image' which focus on generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case: retrieve pricing information for flux-2 models. However, it lacks explicit exclusions or alternative suggestions, falling just short of a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It clearly indicates a read-only fetch operation and specifies the data returned (status and payload), but it does not disclose potential limitations such as pending tasks (no payload yet) or error behavior. This is basic behavior disclosure, not rich context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant wording. It efficiently conveys the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and minimal annotations, so the description must compensate. While it states the high-level return (status and payload), it does not describe the response structure or behaviors like task not found or incomplete results. Given the simplicity of the tool, the description is usable but incomplete for an agent needing to handle edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes both parameters (task_id and action) with descriptions, including the enum for action. The description itself adds no additional parameter semantics, but since schema coverage is 100%, the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and clearly identifies the resource ('current status and latest result payload for a flux-2 task'). It distinguishes from sibling tools like remix_image and text_to_image, which create tasks, by focusing on retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to check on an existing task, but it does not explicitly state when to use it versus the creation tools or any prerequisites (e.g., 'call after creating a task'). The schema hints that action must match the creation endpoint, but the description itself offers no usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must fully disclose behavior. It mentions the PKCE flow and saving to a config file, but does not note potential side effects like overwriting existing credentials or whether the tool checks for existing sessions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that efficiently conveys the tool's operation without unnecessary words. Every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple login tool with one optional parameter and no output schema, the description is largely complete. However, it could mention if the tool handles existing credentials or if the force parameter is needed for re-authentication.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the only parameter 'force'. The description does not add extra meaning beyond the schema's description, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'authenticate' and the resource 'RunAPI', specifying the PKCE login flow and saving the API key to a config file. This distinguishes it from sibling tools that handle unrelated tasks like pricing or image generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives, but the siblings are unrelated so the usage context is implied. Missing details like when to set the 'force' parameter or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remix_imageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the burden. It discloses the asynchronous nature (returns a task id) and outputs, but does not mention auth, costs, or other behavioral traits like polling or safety checker defaults. Adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys purpose and return values without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 11 parameters and no output schema, the description covers the essential return contract (task id, status, output URLs) and the core action. This is sufficient for an asynchronous task tool, though more detail on workflow could improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 82%, which is high, so the schema already documents most parameters. The description adds no extra parameter guidance, but it is not needed given the strong schema coverage. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create', the resource 'Flux 2 task', and the parenthetical 'remix image' distinguishes it from the sibling 'text_to_image'. It also specifies the key return values (task id, status, output URLs), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for image remixing with 'Flux 2' and distinguishes from the text-to-image sibling, but it does not explicitly state when to use this tool versus alternatives or any exclusions. Context is clear but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.