Server Quality Checklist

Tool Scores

• generate_reference2video_to_videoA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by including the cost warning about API calls to Vidu, which is crucial behavioral context not in the schema. It also describes return values ('task_id and video_url'), though output format details are minimal. It doesn't cover error handling, rate limits, or authentication needs, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately front-loaded with the core purpose and cost warning. However, it's lengthy due to detailed parameter explanations, which are necessary given the poor schema coverage. Some redundancy exists (e.g., repeating 'Defaults to' for multiple parameters), and the structure could be tighter, but overall it's reasonably organized for a complex tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 9 parameters, 0% schema coverage, no annotations, and no output schema, the description does a good job of providing necessary context. It covers purpose, cost behavior, parameter semantics, and return values. It lacks details on error cases, rate limits, and full output structure, but given the constraints, it's mostly complete and actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description compensates fully by providing detailed semantic information for all 9 parameters. It explains each parameter's purpose, constraints (e.g., '1 to 3 images', 'maximum length of 1500 characters'), defaults, accepted values, and interdependencies (e.g., model-specific duration and resolution options). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a video from a pic and prompt.' This specifies the verb ('generate') and resource ('video'), though it uses 'pic' instead of the more accurate 'images' from the schema. It distinguishes from siblings like 'generate_text_to_video' by mentioning image input, but doesn't explicitly contrast with all siblings (e.g., 'generate_template_to_video').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance with the 'COST WARNING' section, advising to 'Only use when explicitly requested by the user.' This establishes a specific context for when to invoke the tool. However, it doesn't explicitly mention when to choose this tool over sibling tools (e.g., vs. 'generate_img_to_video'), which would be needed for a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_img_to_videoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it's a generative tool that may incur costs, has model-specific defaults and constraints (e.g., duration and resolution options), and returns a task_id and video_url. It also notes that BGM is only supported for 4-second videos. While comprehensive, it could benefit from mentioning potential rate limits or error handling, keeping it from a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, cost warning, args, returns) and uses bullet points for readability. It's appropriately sized for an 8-parameter tool, with each sentence adding value. However, some redundancy exists (e.g., repeating model names in multiple sections), and the opening sentence could be more front-loaded with key details, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 8 parameters, no annotations, and no output schema, the description is highly complete. It covers purpose, usage warnings, detailed parameter semantics, and return values. The only minor gap is the lack of explicit differentiation from sibling tools, but given the richness of other details, this is a strong effort that nearly meets all contextual needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given a schema description coverage of 0%, the description compensates fully by providing detailed semantics for all 8 parameters. It explains each parameter's purpose, default values, allowed values (e.g., model options, resolution options), and constraints (e.g., prompt length limit, BGM support conditions). This goes well beyond the basic schema, making it easy for an agent to understand how to use each parameter correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a video from a pic and prompt.' It specifies the verb ('generate'), resource ('video'), and input types ('pic and prompt'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'generate_text_to_video' or 'generate_template_to_video', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance with a 'COST WARNING' that advises using the tool only when explicitly requested by the user, which helps prevent unnecessary API calls. It also mentions model-specific defaults and constraints (e.g., BGM only supported for 4-second videos), offering practical context. However, it doesn't explicitly state when to use this tool versus its siblings (e.g., 'generate_text_to_video'), which limits the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_startend2video_to_videoA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively adds value by disclosing cost implications ('COST WARNING: This tool makes an API call to Vidu which may incur costs'), model-specific defaults for duration and resolution, and constraints like BGM only supported for 4-second videos. This goes beyond what the input schema provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized but could be more front-loaded; the cost warning is prominent, but parameter details are listed in a structured 'Args:' section which is clear but somewhat verbose. Every sentence earns its place by providing necessary information, though it might benefit from tighter formatting.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, no annotations, no output schema), the description is largely complete. It covers purpose, usage guidelines, behavioral traits, and detailed parameter semantics. However, it doesn't fully explain the return values ('task_id and video_url') or potential errors, leaving minor gaps in contextual completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description compensates fully by adding detailed semantics for all 8 parameters. It explains the purpose of 'images' as start and end frames, lists model options with defaults, specifies prompt length limits, details duration defaults and options per model, clarifies seed behavior, resolution dependencies, movement amplitude options, and BGM constraints. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a video from a pic and prompt.' It specifies the verb 'generate' and resource 'video,' though it doesn't explicitly differentiate from sibling tools like 'generate_img_to_video' or 'generate_text_to_video.' The mention of 'two images: first is start frame, second is end frame' provides some specificity but not full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a clear usage guideline: 'Only use when explicitly requested by the user' due to cost warnings. It doesn't explicitly state when to use this tool versus alternatives like 'generate_img_to_video' or 'generate_text_to_video,' but the cost warning and parameter details (e.g., requiring two images) imply context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_template_to_videoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively adds context beyond basic functionality: it warns about API costs, specifies that BGM is only supported for 4-second videos, and explains default behaviors (e.g., random seed, aspect ratio defaults). This provides useful operational insights, though it could mention rate limits or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for Args and Returns, making it easy to scan. It is appropriately sized, with each sentence adding value (e.g., cost warning, parameter details). However, some redundancy exists (e.g., repeating 'Defaults to' for multiple parameters), slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, no annotations, no output schema), the description is largely complete. It covers purpose, usage warnings, parameter details, and return values. However, it lacks information on output behavior (e.g., what task_id and video_url represent, error cases), which would be helpful since there's no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It does so by detailing all 8 parameters with clear semantics: it explains what each parameter does (e.g., 'Different templates have different call parameters'), provides constraints (e.g., prompt max length), defaults, and specific usage notes (e.g., area and beast are template-specific). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a video from a template.' It specifies the verb 'generate' and the resource 'video from a template,' distinguishing it from sibling tools like generate_text_to_video or generate_img_to_video by focusing on template-based generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool: 'Only use when explicitly requested by the user.' It also includes a cost warning, which helps in decision-making. However, it does not specify when to use this tool versus its siblings (e.g., generate_img_to_video), so it lacks full alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_text_to_videoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by mentioning the cost implication ('may incur costs'), specifying that it makes an API call to Vidu, and providing important behavioral details like default values, value ranges, and model-specific constraints. However, it doesn't mention rate limits, authentication requirements, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, cost warning, parameters, returns) and efficiently conveys necessary information. While comprehensive, some sentences could be more concise (e.g., the resolution section has repetitive model/duration combinations). Overall, most sentences earn their place by adding value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 9 parameters, no annotations, and no output schema, the description does remarkably well. It covers the core functionality, cost implications, detailed parameter semantics, and return values. The main gap is the lack of output schema explanation - while it mentions 'task_id and video_url', it doesn't describe their format or how to use them. Given the complexity, it's quite complete but not perfect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given that schema description coverage is 0% (no parameter descriptions in the schema), the description compensates excellently by providing comprehensive parameter documentation. It explains each parameter's purpose, default values, acceptable values, model-specific constraints, and important behavioral notes (like BGM only working with 4-second videos). This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a video from a prompt.' This is a specific verb+resource combination that indicates what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'generate_img_to_video' or 'generate_template_to_video' which suggests similar video generation from different inputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides excellent usage guidance with the 'COST WARNING' section that explicitly states when to use the tool ('Only use when explicitly requested by the user') and warns about potential costs. This gives clear context for appropriate usage, though it doesn't mention specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.