Server Quality Checklist

Tool Scores

• scale_videoC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions scaling video but doesn't describe key traits: whether it modifies the original file or creates a new one, what formats are supported, if it preserves aspect ratio or crops, error handling (e.g., for invalid paths), or performance implications. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with a title and bullet points for parameters, making it easy to scan. However, it's under-specified—the title '视频缩放' is too brief, and the parameter list lacks crucial details like 'video_path'. While structured, it could be more informative without adding bulk.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (video processing with 4 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain the operation's effect (e.g., creates a new file), input/output formats, error cases, or how it differs from siblings. For a mutation tool with undocumented parameters, this provides insufficient context for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists three parameters (width, height, output_path) with brief explanations, but misses 'video_path' (a required parameter) and doesn't clarify data types (e.g., width/height as integers vs. strings per schema) or constraints (e.g., positive values, path formats). The added meaning is minimal and incomplete for the 4 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states '视频缩放' (video scaling) which indicates the tool resizes video, providing a basic verb+resource. However, it's vague about the exact operation (e.g., resizing to specific dimensions vs. scaling by percentage) and doesn't distinguish it from sibling tools like 'overlay_video' or 'clip_video' that might also manipulate video dimensions. The purpose is clear but lacks specificity and differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an input video file), exclusions (e.g., not for cropping or overlaying), or comparisons to siblings like 'clip_video' (which might resize by trimming) or 'overlay_video' (which might involve resizing in composition). Usage is implied by the parameters but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_infoC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves information (implying a read-only operation) but doesn't clarify whether it requires specific permissions, has rate limits, returns structured data, or handles errors. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three short sentences, but it's not optimally structured. The purpose statement is clear, but the parameter and return sections are brief and lack depth. While there's no wasted text, it could be more front-loaded with critical details given the lack of annotations and output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a tool with one parameter but no annotations or output schema), the description is incomplete. It doesn't explain the return format ('视频详细信息' - video detailed information is vague), error handling, or behavioral constraints. With no structured fields to rely on, the description should provide more context to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal parameter semantics: it lists 'video_path (str): 输入视频文件路径' (input video file path), which matches the schema's single parameter. However, schema description coverage is 0%, so the schema provides no additional details. The description doesn't elaborate on path formats, supported video types, or validation rules, offering only basic clarification beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '获取视频信息，包括时长，帧率，codec等' (Get video information, including duration, frame rate, codec, etc.). It specifies the verb ('获取' - get) and resource ('视频信息' - video information) with concrete examples of what information is retrieved. However, it doesn't explicitly differentiate from sibling tools like 'find_video_path' which might also retrieve video metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, context for usage, or comparisons to sibling tools like 'find_video_path' (which might find paths) or 'extract_frames_from_video' (which might provide frame details). The agent must infer usage solely from the purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• play_videoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool uses ffplay and supports specific formats, but lacks critical details: whether it blocks execution until playback completes, what happens on errors, if it requires specific permissions or system dependencies, or any side effects. The description covers basic functionality but misses important operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by a bulleted list of parameters with explanations. There's minimal waste, though the format list could be slightly trimmed. Structure is clear with purpose first, then parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, 0% schema coverage, no annotations, and no output schema, the description is moderately complete. It explains parameters well and states the tool's purpose, but lacks behavioral details (e.g., blocking nature, error handling) and usage context. For a playback tool with no structured metadata, it should provide more operational guidance to be fully adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics for all three parameters: video_path is explained as the file path, speed as playback rate with suggested range (0.5-2), and loop as integer with clear values (1=no loop, 0=loop). This goes beyond the schema's basic type information, though it could clarify path format or default behaviors more.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '使用 ffplay 播放视频文件' (use ffplay to play video files) with specific resource (video files) and verb (play). It distinguishes from siblings like clip_video or concat_videos by focusing on playback rather than editing or processing. However, it doesn't explicitly contrast with all siblings (e.g., overlay_video might also involve playback).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions supported formats (mkv, mp4, etc.) but doesn't explain when to choose play_video over other video-related tools like get_video_info or overlay_video. There's no mention of prerequisites, context, or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clip_videoB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool uses ffmpeg for execution and includes a timeout parameter, which adds some behavioral context. However, it lacks critical details: it doesn't specify whether the operation is read-only or destructive (clipping likely modifies or creates files), doesn't mention error handling beyond the error code, and omits information about permissions, rate limits, or side effects on the source video.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately concise but lacks front-loading of key information. It starts with a general title, then lists parameters in detail, which is useful but could be better organized. The inclusion of an example at the end is helpful, but some sentences are redundant (e.g., repeating time format info). Overall, it's adequately sized but not optimally structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no annotations, no output schema), the description is partially complete. It covers parameter semantics well but falls short on behavioral aspects like safety, error details, and output explanation. The example helps, but without annotations or output schema, more context on the return values (e.g., what the error codes mean) would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains each parameter's purpose, formats (e.g., time formats for start/end), defaults, and constraints (e.g., end and duration must have one). This compensates well for the schema's lack of descriptions, though it could be more structured and doesn't fully clarify all parameter interactions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as '智能视频剪辑函数' (intelligent video clipping function) and specifies it clips video files. It distinguishes from siblings like concat_videos or overlay_video by focusing on temporal clipping rather than concatenation or overlaying. However, it doesn't explicitly contrast with extract_frames_from_video which also involves video extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the parameter explanations (e.g., 'end和duration必须有一个' - end and duration must have one), suggesting when to use certain parameters. However, it doesn't provide explicit guidance on when to choose this tool over alternatives like extract_frames_from_video or scale_video, nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_frames_from_videoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the extraction action but lacks critical details: it doesn't mention whether this is a read-only or destructive operation, potential performance impacts (e.g., large video processing), error handling, or output behavior (e.g., file creation confirmation). The description covers basic functionality but misses important behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with a clear purpose statement, followed by a bullet-point-like list of parameters with explanations. Each sentence adds value without redundancy. Minor improvement could be made by front-loading key constraints, but overall it's efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no output schema, no annotations), the description is partially complete. It thoroughly documents parameters but lacks context on behavioral aspects (e.g., side effects, errors) and output details. Without annotations or output schema, users might not understand the full operation scope, though parameter coverage is excellent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value beyond the input schema, which has 0% schema description coverage. It clearly explains all five parameters in Chinese, including 'video_path' (video path), 'fps' (frames per second with examples), 'output_folder' (output directory), 'format' (image format with mappings: 0=PNG, 1=JPG, 2=WEBP), and 'total_frames' (maximum frames, 0=unlimited). This compensates fully for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '提取视频中的图像' (extract frames from video). It specifies the verb (extract) and resource (frames from video), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'clip_video' or 'get_video_info', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., video file accessibility), compare to siblings like 'clip_video' for different operations, or specify scenarios where extraction is appropriate. Usage is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_video_pathB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses key behaviors: recursive search, exact filename matching, extension-agnostic matching, returns first match or empty string. However, it lacks details on permissions needed, error handling, performance (e.g., speed for large directories), or system-specific constraints. The description adds value but is incomplete for a tool with mutation-like file access.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by parameter and return details. It uses bullet-like structure for parameters and return value, making it easy to scan. There's minimal redundancy, though the Chinese phrasing '查找文件路径' is slightly repetitive. Overall, it's efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters, no annotations, and no output schema, the description is moderately complete. It covers purpose, parameters, and return value adequately. However, for a tool that accesses filesystem (implied mutation risk), it lacks safety warnings, error details, or output format beyond '完整路径' (full path). With richer context needed for file operations, it's adequate but has clear gaps in behavioral transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for both parameters: 'root_path - 要搜索的根目录' (root directory to search) and 'video_name - 视频文件名（可以带扩展名，但会忽略扩展名匹配）' (video filename, extension ignored). This clarifies semantics beyond the schema's basic string types, though it could detail format constraints (e.g., path syntax). With 2 parameters fully explained, it compensates well for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '可以查找视频文件路径，查找文件路径，递归查找精确匹配文件名的视频文件路径（支持带或不带扩展名）' - it specifies the verb (find/locate), resource (video file paths), and scope (recursive search with exact filename matching). It distinguishes from siblings like 'get_video_info' (metadata) or 'play_video' (playback), though not explicitly. The purpose is specific but could be more distinct from generic file search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_video_info' for metadata or 'play_video' for playback, nor does it specify prerequisites or exclusions (e.g., when not to use it). Usage is implied by the purpose but lacks explicit context for selection among video-related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• overlay_videoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the picture-in-picture effect but lacks critical details: whether it modifies original files, what permissions are needed, how errors are handled, output format, or performance characteristics. For a video processing tool with 6 parameters, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by a clear parameter section. Each parameter explanation is brief but informative. There's minimal redundancy, though the Chinese-only format might limit accessibility in some contexts.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters, no annotations, and no output schema, the description provides adequate parameter semantics but lacks behavioral context. It covers what the tool does and parameter meanings, but doesn't address mutation behavior, error handling, or output characteristics. For a video processing tool, this leaves important gaps despite good parameter coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides meaningful explanations for all 6 parameters: clarifies 'background_video' and 'overlay_video' roles, explains 'position' enum values with detailed mappings, and defines 'dx' and 'dy' as offset values. This adds substantial value beyond the bare schema, though some details like file format requirements remain unspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '两个视频叠加，注意不是拼接长度，而是画中画效果' (overlay two videos with picture-in-picture effect, not concatenation). It specifies the verb ('叠加' - overlay) and resource (videos), and distinguishes from sibling tools like 'concat_videos' by emphasizing it's not concatenation. However, it doesn't explicitly differentiate from all siblings like 'scale_video' or 'clip_video'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating '画中画效果' (picture-in-picture effect) and contrasting with concatenation, suggesting this tool is for overlaying videos rather than joining them end-to-end. However, it doesn't provide explicit guidance on when to use this versus alternatives like 'scale_video' or 'clip_video', nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• concat_videosA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it uses FFmpeg, returns execution logs, has default output path behavior, describes failure conditions (incompatible parameters when fast=True), and mentions output format determination. However, it doesn't cover rate limits, authentication needs, or what happens to source files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (description, parameters, return, notes), appropriately sized for a 3-parameter tool, and front-loaded with the core purpose. Some redundancy exists (fast parameter details repeated in notes), and the boolean description could be more concise, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with no schema descriptions, no annotations, and no output schema, the description provides good coverage: explains all parameters, return value, technical constraints, and failure conditions. Missing details about error handling formats, performance characteristics, or exact default output path behavior prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining all three parameters: input_files as video file path list, output_path with default behavior, and fast with detailed boolean semantics including technical requirements and when to use each value. It adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('使用FFmpeg拼接多个视频文件' - use FFmpeg to concatenate multiple video files), distinguishes it from siblings like clip_video, overlay_video, or scale_video by focusing on concatenation rather than editing or transformation, and provides the exact resource being manipulated (video files).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (concatenating videos) and includes technical prerequisites (video parameter compatibility), but doesn't explicitly state when NOT to use it versus alternatives like overlay_video for combining videos with effects, or when to choose clip_video for trimming instead. The '注意' section offers some guidance but not explicit alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.