Server Quality Checklist

Tool Scores

• analyze_streamsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool performs '深度分析' (deep analysis) of streams, but doesn't describe what this entails—e.g., whether it's a read-only operation, if it requires specific file permissions, potential performance impacts, or error handling. The description lacks details on behavioral traits beyond the basic function, leaving gaps for an AI agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of a single sentence that directly states the tool's purpose: '分别解析视频流和音频流的详细参数，提供流级别的深度分析'. It avoids unnecessary words and efficiently communicates the core function. However, it could be slightly improved by adding brief usage context without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the context—2 parameters with full schema coverage, no annotations, no output schema, and sibling tools present—the description is minimally adequate. It clarifies the tool's purpose but lacks guidance on usage versus alternatives and behavioral details. For a tool with no output schema and no annotations, more information on expected outputs or operational constraints would enhance completeness, but the current description meets a basic threshold.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for both parameters: 'filePath' (video file path) and 'includeMetadata' (whether to include metadata information). The description adds no additional semantic meaning beyond the schema, such as explaining parameter interactions or usage nuances. Since schema coverage is high, the baseline score of 3 is appropriate, as the schema adequately handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '分别解析视频流和音频流的详细参数，提供流级别的深度分析' (Parse video and audio stream parameters separately, provide stream-level deep analysis). It specifies the verb ('解析' - parse/analyze) and resource ('视频流和音频流' - video and audio streams), making the function clear. However, it doesn't explicitly differentiate from sibling tools like 'get_video_info' or 'calculate_bitrate', which might offer overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools such as 'get_video_info' (which might provide general video information) or 'calculate_bitrate' (which might focus on bitrate calculations), nor does it specify prerequisites or exclusions. Usage is implied only by the tool's name and description, with no explicit context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_bitrateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions calculating average and peak bitrates with detailed analysis, but doesn't cover critical aspects like whether this is a read-only operation, potential performance impacts, error handling, or output format. For a tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core functionality and avoids redundancy, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that performs analysis. It doesn't explain what 'detailed bitrate analysis' includes, such as output format, units, or additional metrics. For a tool with 2 parameters and potential complexity in bitrate calculations, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('filePath' and 'sampleDuration') with descriptions. The description doesn't add any parameter-specific details beyond what the schema provides, such as explaining how 'sampleDuration' affects peak bitrate calculation. Baseline 3 is appropriate when the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '计算视频的平均码率和峰值码率' (calculate average and peak bitrate of videos) and '提供详细的码率分析' (provide detailed bitrate analysis). It specifies the verb (calculate) and resource (video bitrate), though it doesn't explicitly differentiate from sibling tools like 'analyze_streams' or 'get_video_info' which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'analyze_streams' or 'get_video_info', nor does it specify contexts or exclusions for usage. The agent must infer usage based on the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool outputs reports in multiple formats but doesn't describe what the report contains, whether it performs analysis on the video file, if it requires specific file types, or what happens on errors. For a tool that presumably reads and processes video files, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two clear clauses in a single sentence. The first clause states the core purpose, the second adds important functionality about output formats. There's no wasted language, though it could be slightly more structured by separating the format support into its own sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what '视频技术参数' (video technical parameters) includes, what the output looks like for different formats, or any behavioral aspects like file access requirements or error conditions. The agent would need to guess about the tool's behavior and output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, with clear documentation for both parameters: 'filePath' (视频文件路径 - video file path) and 'format' (报告格式 - report format) with enum values. The description adds no additional parameter semantics beyond what's already in the schema, so it meets the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '输出标准化的视频技术参数报告' (output standardized video technical parameter reports) and '支持多种格式输出' (supports multiple output formats). It specifies the verb ('输出' - output) and resource ('报告' - report) with the scope of video technical parameters. However, it doesn't explicitly differentiate from sibling tools like 'get_video_info' which might provide similar information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_video_info' or 'analyze_streams'. It mentions support for multiple output formats but doesn't specify scenarios where one format would be preferred over others, nor does it mention any prerequisites or constraints for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what information is retrieved but doesn't describe how the tool behaves: e.g., whether it reads files safely (non-destructive), handles errors for invalid paths, requires specific permissions, has performance or rate limits, or returns structured data. The description is functional but lacks operational context needed for safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('获取视频文件的详细信息') and lists key attributes. There is no wasted verbiage or redundancy, making it easy to parse. However, it could be slightly more structured by separating usage context or behavioral notes, but it remains appropriately concise for its informational content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with one parameter) and the absence of annotations and output schema, the description is incomplete. It explains what information is retrieved but not the return format, error handling, or operational behavior. For a tool that interacts with file systems, more context on safety, permissions, and output structure is needed to be fully actionable without relying on external assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with 'filePath' clearly documented as '视频文件路径' (video file path). The description adds no additional parameter semantics beyond what the schema provides, such as format examples (e.g., absolute vs. relative paths) or constraints (e.g., supported file types). With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb '获取' (get) and the resource '视频文件的详细信息' (detailed information of video files), listing specific attributes like duration, resolution, frame rate, and encoding. It distinguishes from siblings by focusing on comprehensive metadata extraction rather than stream analysis, bitrate calculation, or report generation. However, it doesn't explicitly differentiate from potential overlapping tools like 'analyze_streams' which might also provide some similar information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'analyze_streams', 'calculate_bitrate', or 'generate_report'. It doesn't mention prerequisites, such as file accessibility or format support, or exclusions for when other tools might be more appropriate. Usage is implied only by the tool's name and purpose, lacking explicit context or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.