Server Quality Checklist

Tool Scores

• quality_scoreA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full transparency burden. It details what metrics are analyzed and what issues are detected, but it does not disclose the output format (e.g., scores, thresholds, reports) or any behavioral constraints (e.g., supported audio duration, processing side effects). This lacks key information for an agent to anticipate the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose followed by concrete details. Every word adds value, with no redundancy or filler. It is highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has moderate complexity, no output schema, and no annotations. The description explains what it does and what it detects, but it omits the return format and any usage limitations. Compared to sibling tools, it lacks differentiation cues and could benefit from explicitly stating output criteria. Overall sufficient but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the single parameter (audio_path) with a description of the path and accepted file types. The tool description adds context that the audio is analyzed for speech quality, but no additional semantics beyond the schema. Baseline of 3 applies since schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes speech quality metrics of an audio file, listing concrete metrics (pitch variation, energy, pacing, silence ratio) and outcomes (detects robotic tone, monotone speech, audio issues). It uses a specific verb and resource, distinguishing it from sibling tools like transcribe (speech-to-text) and compare_tts (comparison).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: if you need to evaluate speech quality, this tool is appropriate. However, it does not explicitly state when to use it versus alternatives like analyze_tts or compare_tts, nor does it mention any exclusions or prerequisites. Guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribeA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It does state the output type ('text with word-level timestamps') and the engine ('Whisper'), but it fails to mention side effects, permissions, network requirements, or whether the tool is read-only. For a tool that processes files, this lack of safety/behavioral detail is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no unnecessary words. It efficiently conveys the action, output details, and underlying model, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description covers the core purpose and output. However, it does not specify the exact return format (e.g., whether it returns a plain text string with timestamps or a structured object), and with no output schema, this ambiguity could confuse an agent. It is not fully complete for invocation but is adequate for a basic tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both 'audio_path' and 'language' described in the schema. The description adds no additional parameter meaning, so the baseline score of 3 applies. It neither enhances nor detracts from the schema's clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb and resource: 'Transcribe an audio file to text with word-level timestamps using Whisper'. It is specific about the output (text with word-level timestamps) and the method (Whisper), distinguishing it from sibling tools like quality_score and compare_tts which focus on evaluation and comparison, not transcription.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you need to convert an audio file to text, use this tool. However, it does not explicitly state when to use it vs. alternatives, nor any exclusions or prerequisites. No mention of sibling tools or scenarios is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_ttsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must shoulder the burden of behavioral disclosure. It states that the tool returns a comprehensive report and that mispronunciation detection is optional, but does not mention potential side effects, performance implications, or any constraints. The read-only nature is implied but not explicitly stated, and no error/edge-case behavior is disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the key phrase 'Full TTS audio analysis,' and efficiently enumerates capabilities without fluff. It is a single well-structured sentence that conveys all essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description provides a reasonable overview of what the tool returns ('comprehensive report') and the optional input that triggers extra functionality. It lacks details on report structure, error handling, or prerequisites (e.g., dependencies), but given the simple parameter set and the presence of sibling tools for narrow tasks, it is sufficiently complete for initial selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage with clear descriptions for each parameter (audio_path, language, expected_text). The description adds no extra parameter information beyond what the schema already offers, such as the relationship between expected_text and mispronunciation detection, which is already in the schema. Thus, the description does not add value beyond the schema, warranting the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Full TTS audio analysis — transcription, quality scores, pacing analysis, and optional mispronunciation detection.' It enumerates specific output components and a use case ('for debugging TTS issues'), distinguishing itself from sibling tools like transcribe and quality_score by combining multiple analyses into one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a usage context ('for debugging TTS issues') but does not explicitly contrast this tool with its siblings. It provides clear context for when to choose this comprehensive analysis over individual tools, but lacks explicit 'when not to use' or alternative recommendations, preventing a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_ttsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the analytical outputs (mispronunciations, inserted/deleted words, WER), giving insight into what the tool returns. It does not explicitly state whether the tool is read-only or mention side effects, but the comparison nature implies non-destructive use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and then lists the key outputs. There is no unnecessary information or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, with all parameters documented in the schema and no output schema present. The description explains what the tool identifies but does not specify the return format or potential limitations. However, given the straightforward purpose and the listed output types, it provides sufficient context for an agent to decide when to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes all three parameters (audio_path, expected_text, language) with 100% coverage. The description adds no additional parameter semantics beyond what the schema already provides, so it stays at the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: comparing TTS audio output against expected text, and specifies the exact outputs (mispronunciations, inserted/deleted words, WER). This distinguishes it from sibling tools like `transcribe` (which does not compare) and `quality_score` (which does not use expected text).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when you have a TTS audio file and the expected text to compare, providing clear context. It does not explicitly mention alternatives or say when not to use it, but the purpose is distinct enough to guide tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.