Server Quality Checklist

Tool Scores

• omni_video_generate_vfxA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full weight. It discloses the use of Hyperframes and output format, but lacks details on side effects, authorization needs, or error behaviors. Adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and output. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple and has an output schema. The description covers purpose and basic behavior but omits when to use, parameter details, and potential prerequisites. Adequate but with gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has one parameter (request) with a nested object that is described in the schema's $defs. However, the tool description does not mention parameters or add meaning beyond the schema. With 0% schema description coverage for the top-level parameter, the description fails to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Renders motion graphics) and resource (e.g., lower thirds, titles) and specifies the output (path to .mov or .webm). It effectively distinguishes from sibling tools like omni_video_ingest, preview, and render.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for motion graphics generation but does not provide explicit guidance on when to use this tool versus siblings, nor does it list prerequisites or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omni_video_renderA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It lists applied operations and notes return of a path, but does not disclose side effects like file writing at output_path, auth requirements, or potential destructive behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence efficiently lists the tool's actions and return value. It is front-loaded with the verb 'orchestrates'. Could be slightly improved by separating the list for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description covers the high-level pipeline and key parameters. It lacks details on EDL JSON structure or output schema, but the separate schema and output schema exist to fill gaps, making it nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description maps 'EDL cuts' to edl_json, 'LUT color grading' to lut_path, and 'subtitle burning' to subtitle_style, adding context beyond the schema. However, it omits output_path and mentions overlay graphics/audio restoration not directly reflected in parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool orchestrates the final render pipeline, listing specific operations (EDL cuts, overlay graphics, LUT grading, audio restoration, subtitle burning) and returns the output path. It distinctly separates from sibling tools like omni_video_preview by focusing on the final render stage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage as the final render step after other pipeline stages, but lacks explicit when-to-use or when-not-to-use guidance, particularly differentiating from omni_video_preview which may serve as a preview step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omni_video_ingestA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It states that the tool ingests video, generates transcripts and scene graphs, and returns a metadata path. However, it does not mention whether files are modified, destructive potential, authentication needs, or time/resource consumption. The description is adequate but lacks depth for a full behavioral profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose, operations, and output. Every phrase adds value: ingestion, transcript generation, scene graph construction, B-roll searching goal, and return of metadata path. No extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main steps and purpose, referencing the output (metadata path) which is further defined by an output schema. It is clear relative to sibling tools. Minor gaps include lack of supported video formats or directory considerations, but overall sufficient for understanding core functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter, directory_path, with a description in the schema itself. The tool description adds that the directory contains 'raw video footage' and that ingestion produces transcript and scene graph, but it does not provide additional semantics beyond what the schema already states. Baseline score of 3 is appropriate given high schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Ingests', 'generates', 'constructs') and resources ('directory of video files', 'audio transcripts', 'Visual Scene Graph') to clearly state the tool's function. It also distinguishes from siblings (generate_vfx, preview, render) by naming the distinct outputs for B-Roll searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used for ingesting video to support B-roll searching, but it does not explicitly state when to use or avoid it versus alternatives like omni_video_generate_vfx or omni_video_preview. No prerequisites or excluded cases are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omni_video_previewA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses return value (absolute path to PNG) but omits side effects, error conditions, performance implications, or what happens with invalid inputs. Minimal behavioral disclosure beyond output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, each adding unique value: generation action, use case, return information. No wasted words, front-loaded with core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present (implied) and comprehensive parameter descriptions, the tool is mostly complete. Could mention error handling or performance constraints for large videos. Still strong given context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all parameters (file_path, start_time, end_time) achieving 100% coverage. The description adds no additional parameter semantics beyond noting the return type. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'generates' and resource 'filmstrip PNG of the specified video segment'. Intent is further clarified with use case for verifying cut boundaries or B-roll placement. Distinguishes from sibling tools (generate_vfx, ingest, render) by focusing on preview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides explicit use cases: 'visually verifying cut boundaries or B-roll placement'. Does not explicitly state when not to use or mention alternatives, but context with sibling tools makes the purpose clear. Slightly improved by adding 'when not to use'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.