Server Quality Checklist

Tool Scores

• get_supported_languagesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool returns a list, which implies a read-only operation, but doesn't disclose behavioral traits like whether it requires authentication, has rate limits, returns structured data, or caches results. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences: one stating the purpose and one detailing the return value. It's front-loaded and wastes no words. However, the structure could be slightly improved by combining ideas more fluidly, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, simple read operation) and the presence of an output schema (which handles return values), the description is minimally adequate. It covers what the tool does and what it returns, but lacks context on usage, behavior, or integration with siblings, leaving gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100% (empty schema). The description doesn't need to add parameter semantics, so it meets the baseline for no parameters. It appropriately focuses on the output rather than inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'サポートされている言語の一覧を取得します' (Get a list of supported languages) and specifies what it returns: '翻訳可能な言語コードと言語名のリストを返します' (Returns a list of translatable language codes and language names). It distinguishes from sibling tools (transcribe_from_file, transcribe_from_url) by focusing on language metadata rather than transcription. However, it doesn't explicitly contrast with siblings beyond the different domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, timing, or relationships with sibling tools. The only implied usage is when language information is needed, but this is too vague for effective tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_from_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by explaining key behavioral traits: it generates SRT files (output format), mentions hallucination prevention for 'condition_on_previous_text' and 'vad_filter', describes language auto-detection and translation capabilities, and indicates default values for many parameters. However, it doesn't cover performance characteristics, error conditions, or file format requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and front-loads the core purpose. Each parameter explanation is concise yet informative. However, some explanations could be more streamlined (e.g., 'condition_on_previous_text' and 'vad_filter' both mention 'ハルシネーション防止' - hallucination prevention, creating slight redundancy).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (11 parameters, no annotations, but with output schema), the description is quite complete. It explains the tool's purpose, all parameters with semantics, and mentions the return values (SRT file path and language info). The output schema existence means the description doesn't need to detail return structure. It could improve by mentioning performance implications or file size limits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage (titles only, no descriptions), the description provides comprehensive parameter semantics beyond what the schema offers. Each of the 11 parameters receives clear explanations in Japanese that add meaning: 'file_path' as absolute path, 'device' for inference hardware, 'model_size' as Whisper model, language codes for input/output, 'initial_prompt' for technical terms, boolean flags with purposes, and thresholds with their functions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'ローカルの動画/音声ファイルから字幕を生成します' (Generate subtitles from local video/audio files). It specifies the verb ('生成します' - generate), resource ('字幕' - subtitles), and source ('ローカルの動画/音声ファイル' - local video/audio files), which distinguishes it from the sibling 'transcribe_from_url' that works with URLs instead of local files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the parameter explanations (e.g., 'device' for CUDA/CPU inference, 'input_lang' for auto-detection when omitted), but doesn't explicitly state when to use this tool versus alternatives like 'transcribe_from_url'. The distinction from the sibling tool is clear (local files vs. URLs), but no explicit guidance on choosing between them is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_from_urlA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by explaining key behaviors: downloading videos, generating subtitles, language detection/translation capabilities, hallucination prevention mechanisms, and return values. It mentions 'ハルシネーション防止' (hallucination prevention) for two parameters, which is valuable behavioral context not evident from schema alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and each parameter explanation is concise yet informative. While comprehensive, it maintains efficiency - every sentence serves a purpose in explaining the tool's functionality or parameters. Minor deduction because the purpose statement could be slightly more front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 parameters, video processing), no annotations, but with output schema present, the description provides excellent completeness. It explains the core functionality, all parameters with semantics, behavioral traits like hallucination prevention, and mentions return values. The output schema handles return format details, so the description appropriately focuses on usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear semantic explanations for all 11 parameters. Each parameter gets a meaningful description in Japanese that explains its purpose, defaults, and practical implications (e.g., '専門用語や固有名詞のヒント' for initial_prompt, 'ハルシネーション防止' for condition_on_previous_text). This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'URLから動画をダウンロードして字幕を生成します' (downloads video from URL and generates subtitles). It specifies the verb (download and generate), resource (video from URL), and distinguishes from sibling 'transcribe_from_file' by specifying URL input rather than file input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through parameter explanations (e.g., 'YouTube等' suggests video platforms), but doesn't explicitly state when to use this tool versus 'transcribe_from_file' or 'get_supported_languages'. It provides technical context but lacks explicit guidance on tool selection scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.