Server Quality Checklist

Tool Scores

• short_video_agent_manifestC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds the context of being machine-readable but does not elaborate on behavioral traits beyond what annotations cover, such as side effects or response characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise (one short sentence), but it lacks essential details about parameters and usage. It is front-loaded but not fully informative, earning a middle score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 optional parameters, no output schema, and annotations that cover safety, the description is incomplete. It fails to explain the purpose of parameters or provide enough context for an agent to use the tool effectively without additional knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not mention the parameters 'client' or 'response_format' at all. With 0% schema description coverage, the description completely fails to add meaning beyond the schema, leaving an agent without guidance on how to use these optional parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides machine-readable install, provider, runtime, and safety guidance, which is specific. However, it lacks an explicit verb like 'get' or 'retrieve', and does not directly distinguish from sibling tools, though the resource is sufficiently clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like short_video_build_payload or short_video_privacy_audit. No when-to-use or when-not-to-use information is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• short_video_privacy_auditC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe, non-destructive read operation. The description adds that it returns safety boundaries but does not provide additional behavioral context such as response format or error behavior. With strong annotations, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (one sentence) but its structure is somewhat awkward; the phrase 'safety boundaries' is not clarified, and the list of items is run-on. It earns its place but could be more precise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with only one optional parameter and no output schema, the description could be more complete by mentioning the response_format parameter and the structure of the returned safety boundaries. Currently, the agent must guess the return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (response_format) with 0% description coverage, and the tool description does not mention or explain this parameter, leaving the agent to infer its purpose from the enum values alone. This is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a verb ('Return') and identifies the resource ('safety boundaries' for prompt, asset, output, provider-key), distinguishing it from sibling tools that deal with manifest, payload, connection, and generation. However, the term 'safety boundaries' is somewhat ambiguous, leaving some room for interpretation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus the sibling tools; the context is implied by the tool name but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• short_video_build_payloadB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds that no provider API is called, reinforcing safety. However, no extra behaviors (e.g., error handling, output format) are disclosed, so the description adds moderate value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy or filler. It conveys the core functionality efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 14 parameters, no output schema, and no parameter descriptions. The description does not explain what the payload is, how it varies by provider, or what to do with the output. The tool is complex, yet the description provides barely any context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description provides no explanation of parameters. With 14 parameters including enums and defaults, the absence of parameter semantics leaves the agent with no guidance beyond schema types and defaults. This is a critical gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (build) and the resource (provider-specific request payload). It also specifies that no provider API call is made, which sets it apart. However, it does not explicitly differentiate from sibling tools like short_video_generate, though the implication is strong.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when preparing a payload without executing it, but provides no explicit guidance on when to use versus alternatives (e.g., short_video_generate). It does not mention prerequisites, exclusions, or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• short_video_generateB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and destructiveHint=false, and the description adds the key behavioral trait that it returns a dry-run payload unless live=true. This is helpful but does not elaborate on other side effects, permissions, or limitations. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (two sentences) with no wasted words, which is good for conciseness. However, given the tool's complexity (14 parameters), it sacrifices necessary detail. It could be restructured to front-load the key behavior while adding parameter hints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 14 parameters, no output schema, and no parameter descriptions, the description is grossly incomplete. It does not explain response format, provider options, size constraints, or how to use the generated payload. For a video generation tool, this leaves major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the schema provides no parameter descriptions. The description only explains the 'live' parameter and env var, ignoring the other 13 parameters. For a tool with many parameters including enums, this is insufficient to guide correct usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it generates a vertical video, providing a specific verb and resource. However, it does not differentiate from sibling tools like short_video_build_payload or short_video_agent_manifest, which may have overlapping purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the dry-run behavior and how to enable live generation via 'live=true' or env var, offering some usage context. However, it gives no guidance on when to use this tool versus siblings, nor any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• short_video_connection_statusB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior. The description adds the note about not exposing API keys, which is helpful but does not significantly extend beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no filler, efficiently communicates the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, and the description does not describe what the response contains. Given sibling tools like short_video_agent_manifest, more context on when to use this specific check would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has one parameter with enum and default, which is self-explanatory. Description does not explain the parameter or its values, and schema description coverage is 0%, so the description misses an opportunity to add value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks configured providers and dry-run mode, which is specific and distinct from sibling tools like short_video_generate or short_video_build_payload.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs. alternatives (e.g., when a status check is needed rather than generation or audit). No prerequisites or exclusions mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.