Server Quality Checklist

Tool Scores

• text_to_videoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns a task id, status, and output URLs, but omits critical behavioral traits: whether the operation is synchronous or asynchronous (wait parameter suggests polling), potential for long execution, authentication requirements, rate limits, or failure modes. The agent is left with insufficient transparency for a task-creation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no waste, and the action is front-loaded. It is concise, but for a tool with 22 parameters, the brevity borders on under-specification. It earns its place but does not leverage the available sentence to add value beyond the one-line purpose. Score 4 for appropriate conciseness without penalizing the lack of detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the high complexity (22 parameters), absence of output schema, and zero annotations, the description is severely inadequate. It only states the return type but does not explain parameter constraints, default behaviors, or edge cases. An agent cannot correctly invoke this tool without external knowledge, making the description functionally incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 18%, meaning 82% of parameters lack any explanation. The description does not compensate for this: it fails to explain any parameter semantics, such as the meaning of wait, seed, aspect_ratio, or the many reference inputs. It only provides high-level context ('text to video') without mapping to specific parameters, leaving the agent to guess the purpose of most fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action (Create a Seedance task), the resource (RunAPI), and the domain (text to video). It distinguishes itself from sibling tools (login, get_task, check_pricing) by stating a unique verb and resource. The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for text-to-video generation but provides no explicit guidance on when to use this tool versus alternatives. It does not mention exclusions or conditions, and the lack of a direct comparison to siblings leaves the agent to infer applicability. No misleading guidance, but also no helpful routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It states it fetches status and payload, but does not mention read-only nature, error behavior for missing tasks, or any side effects. This is insufficient for a simple fetch tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no redundant information. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description vaguely mentions 'status and latest result payload' but lacks detail on return format or structure. For a fetch tool, more specificity about the response would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both parameters described), so the description adds no extra meaning beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Fetch' and clearly names the resource 'seedance task', and the purpose is distinct from sibling tools (login, text_to_video, check_pricing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after task creation but does not explicitly state when to use it versus alternatives like text_to_video or what conditions are required. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_pricingA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. 'Look up' implies a read-only operation, but the description does not explicitly state side effects, response behavior, authentication needs, or error cases. It is minimally transparent but not richly detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundant information. Every word contributes to the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity and fully documented parameters, the description is mostly complete. However, there is no output schema and the description does not explicitly state what the returned pricing information looks like, which is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters with enums and defaults. The description adds no additional parameter meaning beyond naming the model line, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Look up') and resource ('RunAPI pricing for the seedance model line'), clearly distinguishing this tool from siblings like text_to_video, get_task, and login. It precisely identifies what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when pricing information for seedance models is needed, but it provides no explicit guidance about when to prefer it over alternatives or any exclusions. It is adequate but lacks direct usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors: opens a browser, uses PKCE flow, saves API key to config file. It also explains the force parameter behavior. However, it does not mention potential side effects like overwriting existing config or prerequisites like browser availability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, complete sentence that efficiently conveys the tool's purpose and method without extra words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema), the description is adequately complete. It explains the authentication flow and where the key is saved. Minor omission: no mention of prerequisites or return behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the 'force' parameter already well-described in the schema. The tool description does not add additional semantics beyond what the schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Authenticate', the resource 'RunAPI', and the method 'PKCE login flow', distinguishing it from sibling tools like check_pricing or text_to_video.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for authentication but does not explicitly state when to use it versus alternatives, nor when not to use it. The 'force' parameter provides some context but no explicit guidance on when to choose this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.