Server Quality Checklist

Tool Scores

• create_scheduleA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses a key behavioral trait: the Agent varies hook angle and structure each run to avoid repetition. However, it does not cover other important aspects such as idempotency, effects on existing schedules, required permissions, or error handling. Since no annotations are provided, the description carries full burden but only partially meets it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, efficiently covering purpose, frequency options, and a notable behavioral detail. Every sentence adds value with no fluff. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity (5 params, no output schema), the description provides sufficient context for setup but lacks information about return values (e.g., schedule ID) or error conditions. The schema covers parameters well, so the description completes the picture reasonably for its purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 5 parameters have schema descriptions resulting in 100% coverage. The tool description adds minimal extra meaning beyond the schema (e.g., noting the Knowledge Base as a source). Per the rubric, baseline is 3 when coverage is high, and the description does not significantly augment the parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to set up an automated recurring video generation and export schedule. It specifies the resource (DemoStudio Agent), verb (set up), and key attributes (frequency options, variation in hook/structure). This clearly distinguishes it from sibling tools like generate_video (one-time) and get_video_status (status check).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives, nor are prerequisites or exclusions mentioned. The sibling context implies scheduling is for recurring generation, but the description does not explicitly state this or advise against using it for one-off tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_videoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description outlines the workflow: writes script, renders scenes, adds voiceover and music, returns a project URL. However, with no annotations provided, it lacks details on authentication, rate limits, or potential side effects, which would enhance transparency. The description is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, front-loaded with the primary action, and each sentence adds value. It efficiently conveys the tool's purpose, process, and usage guidance without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, creative generation) and lack of output schema, the description adequately explains what the tool does and that it returns a project URL for preview and export. It could mention processing time or limitations, but overall it provides sufficient context for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema's parameter descriptions are already detailed (e.g., prompt includes a full creative brief). The main description does not add significant new information about parameters beyond what the schema provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates short-form demo videos for specific platforms (Instagram Reel/TikTok/YouTube Shorts) from a text prompt. It distinctly differentiates from sibling tools like create_schedule, get_video_status, and list_capabilities by focusing on video creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when the user wants to create a promotional video, product demo, or UGC-style ad.' This provides clear context for when to use the tool, though it doesn't mention when not to use it or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the return values (status, scene build progress, URL) but does not mention error handling or states like invalid project_id. Given no annotations, it provides adequate transparency for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no extraneous words. Every sentence adds value, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential return data and source of the project_id. However, it does not mention that the tool is intended for polling or that progress might be incomplete; still, it is sufficient for a simple status check.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the parameter fully (project_id as a UUID from generate_video). The description adds minimal additional context ('by its ID') and does not enhance understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Check' and the resource 'status of a video project by its ID'. It distinguishes from sibling tools like generate_video and create_schedule by focusing on status retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after generating a video, but does not explicitly mention when not to use or provide alternatives. The context from sibling tools helps, but it lacks explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_capabilitiesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It specifies exact counts (5 scene types, 17 templates) and implies read-only behavior. No contradictions or missing critical behavioral info.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, purpose stated first, usage guidance and specifics follow. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No missing info for a parameterless listing tool. Description covers purpose, usage context, and return content. Sibling tools are unrelated, so no confusion.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has 0 parameters and schema coverage is 100% (empty schema). Description adds no parameter info, but none is needed. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it lists 'scene types and Remotion motion-graphic templates', uses specific verb 'List' and resource 'DemoStudio', and clearly distinguishes from sibling tools that create schedules, generate videos, or get status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Call this before writing a detailed prompt so you can reference the correct scene types and template names.' No exclusions or alternatives mentioned, but sibling tools are clearly different in function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.