Server Quality Checklist

Tool Scores

• text_to_videoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses return values (task id, status, output URLs) but fails to mention asynchronous behavior, polling, or blocking semantics, despite parameters like wait, timeout_ms, and poll_interval_ms that indicate such behavior. This is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and contains no fluff. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters, no annotations, no output schema, and low schema description coverage, the description is too minimal. It does not explain the asynchronous task model, key parameters, or how this tool interacts with get_task, leaving the context incomplete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 22% (2 of 9 properties have descriptions), so the description must compensate. However, the description adds no parameter-level information, leaving the agent without guidance on key fields like prompt, model, or wait. This is a serious deficiency.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Create' and identifies the resource 'Hailuo task on RunAPI', explicitly noting 'text to video'. It clearly distinguishes from siblings like image_to_video and get_task by stating the action and the return type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for text-to-video generation but does not explicitly state when to use this tool versus alternatives such as image_to_video, or when to follow up with get_task to retrieve results. No exclusions or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose any behavioral traits such as rate limits, authentication needs, or side effects. For a lookup tool, it is a safe read operation, but the description does not explicitly state that it is read-only or non-destructive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the purpose. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and only 2 parameters with full schema coverage. The description is minimally adequate but does not explain return values or behavior when parameters are omitted. For a simple lookup, it is functional but lacks depth.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with both parameters having enum-based descriptions. The description adds minimal meaning beyond the schema by reiterating the 'hailuo' model line, which is already in the enum values. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the verb 'look up' and the resource 'RunAPI pricing', scoped to 'the hailuo model line'. It distinguishes clearly from siblings like get_task, image_to_video, and text_to_video, which are about task retrieval and video generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It does not provide conditions, exclusions, or mention any prerequisites. The description lacks context for proper tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• image_to_videoA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does disclose that the tool returns a task id, status, and output URLs. However, it does not disclose side effects, asynchronous behavior, cost implications, or failure modes, which is a significant gap for a creation task.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the purpose and return values without any wasted words. It earns its place and is easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 12 parameters, no output schema, and no annotations, this description is too minimal. It does not explain task lifecycle, how wait/polling works, model selection, or output format expectations, leaving the agent with substantial gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 17% and the description adds no parameter-level detail. It only hints at the nature of the tool ('image to video'), which indirectly suggests first_frame_image_url, but does not explain wait, model, timeout, callback, resolution, or other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Create a Hailuo task on RunAPI (image to video)', specifying both the resource and the mode. It implicitly differentiates from sibling text_to_video by explicitly noting 'image to video'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for image-to-video generation, which distinguishes it from text_to_video, but it does not explicitly state when to choose this tool over alternatives or mention any exclusions or prerequisites. Usage context is only lightly implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full disclosure burden. 'Fetch' clearly indicates a read operation, and it mentions the status and latest result payload, but it does not disclose behavioral traits such as what happens while a task is still processing, whether authentication is required, or any polling behavior. It is not misleading but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 12 words, front-loaded with the verb and object. Every word contributes value, with no repetition or fluff, making it optimally concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only 2 parameters, but no output schema and no annotations. The description explains the basic function but does not provide context about the asynchronous nature of the task creation or how the result payload is structured. It is adequate for a simple status fetch but leaves some contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage, with both parameters described ('Task id returned when the task was created' and 'Asynchronous endpoint the task was created on'). The description adds no additional parameter semantics beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Fetch' and clearly identifies the resource as 'current status and latest result payload for a hailuo task.' It distinguishes itself from siblings like image_to_video and text_to_video, which are task-creation tools, by being the retrieval/status-check tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after a task has been created via image_to_video or text_to_video, but it does not explicitly state this or mention alternatives. There is no 'when to use' or 'when not to use' guidance, so the usage context is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that a browser login flow is initiated and that an API key is saved to a specific file path. However, it does not mention side effects like overwriting an existing key or network requirements. With no annotations provided, this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately conveys the tool's purpose and method. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple authentication tool with one optional parameter and no output schema, the description covers the essential behavior. However, it could be more complete by explaining the 'force' parameter and what happens if already authenticated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description does not add any extra meaning beyond the schema for the 'force' parameter. The baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Authenticate') and clearly identifies the resource ('RunAPI') and the authentication method (PKCE login flow). It distinguishes from sibling tools, which are unrelated (e.g., check_pricing, text_to_video).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly suggests the tool is for initial authentication, but it does not provide explicit guidance on when to use it versus alternatives, nor does it mention any prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.