Server Quality Checklist

Tool Scores

• generate_subtitlesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden. It discloses that two files are saved when both native and translation are requested and mentions input format support ('.3gp and .ts'). However, it omits other behavioral details like whether existing files are overwritten, what the tool actually returns, or background-mode limitations, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is six sentences long, front-loaded with a clear purpose, and uses selective examples. It avoids fluff but could be tightened without losing scope. Each sentence adds operational value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's 14 parameters, no output schema, and no annotations, the description should clearly explain what the tool returns (e.g., saved file paths) and any critical behavior like overwriting. It does neither. It also doesn't mention background mode viability, leaving the agent without enough context to predict outcomes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description repeats a few parameter highlights (language='auto', translate_to_english=true) but adds no new meaning beyond the schema's already detailed parameter descriptions. It contributes only playback-format context, not parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Generate subtitle files for an audio or video file using whisper.cpp', clearly stating a specific verb, resource, and tool. It further details output formats (SRT/VTT) and translation capability, distinguishing it from siblings like transcribe_audio.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete usage tips: setting language='auto', enabling translate_to_english, and choosing SRT vs VTT based on playback context. However, it does not explicitly contrast this tool with alternatives such as transcribe_audio, leaving selection criteria implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_mediaA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explains behavior for single files and folders, including the sorted table output. However, it lacks details on supported media formats, error handling, and side effects. No annotations are provided, so the description carries the full burden; more detail on limitations or prerequisites would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences, each serving a distinct purpose: stating the tool's function, detailing behavior for single/folder, and suggesting use cases. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers single and folder scenarios, use cases, and mentions FFprobe. Given the lack of output schema and annotations, it provides sufficient context for an agent to decide when to use this tool. Missing details like supported formats or accuracy of time estimates are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for both parameters (path and sort_by) with 100% coverage. The description adds context about the path being an absolute Windows path and the sort_by default, but this adds only marginal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes media files using FFprobe before transcribing, returning specific metadata like duration, size, codec, and estimated transcription times. It distinguishes from sibling tools (e.g., transcribe_audio, check_progress) by focusing on pre-transcription analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to plan batch work, estimate how long transcription will take, or check what's already been transcribed,' providing clear use cases. It does not explicitly state when not to use it, but the context implies it is for planning, not actual transcription.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_systemA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It describes a non-destructive detection/reporting action, but does not explicitly state it is read-only or guarantee no side effects. The implication is safe, but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence that front-loads the purpose ('Detect GPU hardware and verify Vulkan acceleration is available') and then lists specific outputs. Every phrase adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains what the tool returns (GPU name, VRAM, Vulkan binary status, recommended Whisper model). It would benefit from mentioning format or structure, but is sufficient for a zero-parameter check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description does not need to explain any. The baseline for no parameters is 4, and the description adds no confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it detects GPU hardware and Vulkan acceleration, reports specific details (GPU name, VRAM, Vulkan binary), and recommends a Whisper model. This distinguishes it from all siblings, none of which perform system checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it should be used before other tools that utilize GPU/Whisper, but does not explicitly state when to use it versus alternatives or provide any exclusions. Given no overlapping siblings, the context is clear but lacks guidance on when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• switch_modelA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behavior: session-scoped change and non-persistence. Lacks details on error handling or what happens with invalid model names. Without annotations, the description is adequate but not fully comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with main purpose. No unnecessary words; each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers essential aspects: purpose, input, and scope. Lacks output specification, but for a simple action with no output schema, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description repeats the same information as the schema. No additional meaning is added, so baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool's purpose: switching the active Whisper model without restarting Claude Desktop. The verb 'switch' and resource 'active Whisper model' are specific. It distinguishes from sibling tools that deal with listing, downloading, or transcribing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on input (model filename or path, must be installed) and session scope. Does not mention alternatives or when not to use, but context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_audioA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: automatic FFmpeg conversion, output default to timestamps, background job behavior, and privacy/confirmation requirements. It also warns that transcript text is processed by Claude's API. These go beyond the schema and provide meaningful operational context, though it stops short of detailing error handling or return structures.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but every sentence carries useful information: file support, conversion, output format, background jobs, and privacy. It is front-loaded with the core purpose and uses a clear warning for privacy. Slightly dense but well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (24 parameters, no output schema), the description covers key contextual aspects: file compatibility, background execution, and privacy. It does not describe the return format beyond the timestamps default, but with the schema's output_format enum, that is partially covered. Overall, it provides enough context for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value for background and privacy_mode by explaining their purpose and global env override, but it does not systematically enhance understanding of the 24 parameters. Most parameter semantics come from the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool transcribes a single audio or video file using whisper.cpp on Windows. The verb 'transcribe' and the resource 'single audio or video file' are specific, and it distinguishes itself from siblings like transcribe_batch by noting it handles a single file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for long files (background=true and check_progress), privacy mode usage, and notes automatic FFmpeg conversion, implying no manual pre-processing. It does not explicitly mention alternatives like transcribe_batch, but the 'single file' wording conveys when to use this tool over the batch variant.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_configA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the components checked but omits details on error behavior or return format. The check is likely non-destructive, but not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. The first sentence states the action, and the second provides usage guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description covers the what and when adequately. Minor omission: no mention of return value or status reporting, but acceptable for this simple check.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, and description coverage is 100% trivial. The description adds value by listing exactly what is verified, which is essential context beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: verifying the availability of whisper-cli.exe, model, and FFmpeg. It uses a specific verb ('Verify') and resource list, distinguishing it from sibling tools like 'check_system' or 'analyze_media'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to 'Run this first if anything fails,' providing a clear usage context. While it does not enumerate alternatives, the directive is strong and practical for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_modelA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It discloses automatic download and trusted namespaces, but fails to mention potential side effects like overwriting existing files, network dependency, or blocking behavior. This is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, input/behavior, security+follow-up. No unnecessary words. Well front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers the core aspects: what it does, what it takes, restrictions, and next step. Lacks error handling info, but that is acceptable for such a tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, baseline is 3. The description adds value by specifying that the download is automatic and restricted to trusted namespaces, which goes beyond the schema. The example model names and reference to list_models enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (download), resource (Whisper model from Hugging Face), and destination (models directory). It distinguishes itself from siblings like list_models and switch_model by specifying its role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides good usage context: it gives examples of model names, mentions filtering to trusted namespaces, and directs to use switch_model afterwards. It implicitly suggests checking list_models first, but does not explicitly state when not to download.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_batch_progressA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description discloses key behavior: automatic advancement to next file and polling requirement. Lacks details on side effects or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, no wasted words. Each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple tool with one parameter; description explains returns (progress, current file, failed files, elapsed time) sufficiently for an agent to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with a single parameter described as 'Batch ID returned by start_batch.' Description adds no extra meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Check' and the resource 'batch progress', distinguishing it from sibling tools like analyze_media or check_config.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call repeatedly until complete and describes automatic advancement, but does not explicitly state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_progressA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that transcript text is processed by Claude's API, explains privacy_mode behavior (metadata only), and describes return fields. No annotations present, so description carries full burden; it covers key behavioral aspects well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, then usage pattern, then privacy warning. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description clearly states return fields. Covers polling pattern, privacy concerns, and parameter usage. Complete for a simple polling tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage, but description adds value by linking job_id to transcribe_audio, explaining privacy_mode overrides, and detailing its effect (metadata only).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks the status of a background transcription job, lists specific return fields (progress, elapsed time, last processed timestamp, transcript), and implicitly distinguishes from siblings that check other things like batch progress or system config.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call repeatedly until complete or failed, and explains privacy_mode usage. Does not explicitly contrast with alternatives like check_batch_progress, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it reads local filesystem only, no network calls, and provides specific output fields. Lacks mention of permissions or side effects, but as a read-only operation, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states purpose, second details output. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations or output schema, description fully explains what the tool does and what output to expect. Covers all essential aspects for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so schema coverage is 100%. Description adds value by detailing output fields beyond schema, which is helpful for agents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb (list), resource (Whisper model files), and specific output details (filename, size, active, quantization, use case). Distinguishes from sibling tools like download_model and switch_model by focusing on listing installed models.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies when to use: before downloading or switching models, as it lists what is already installed. No explicit when-not or alternatives, but context is clear given sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_batchA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite having no annotations, the description thoroughly discloses behavior: previewing, waiting for confirmation, saving .txt files next to sources, skipping already transcribed files, supported formats, and privacy mode effects. This exceeds typical transparency expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively lengthy but well-structured, with each sentence adding value (formats list, alternative tool, privacy warning). It is not overly verbose, but could be tightened slightly by moving the format list to an appendix.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers the workflow (preview, confirmation, saving, skipping) and privacy considerations. However, it does not describe what the tool returns besides 'metadata only' in privacy mode, leaving a minor gap for interactive completion behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context for privacy_mode (suppress previews, return metadata) but does not materially enhance understanding of other parameters beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool transcribes multiple audio/video files in a folder, with specific behavioral details (one file at a time, preview, confirmation). It also distinguishes itself from the sibling tool start_batch, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use this tool vs. an alternative: 'For large unattended batch jobs, use start_batch instead.' Also describes interactive context and privacy mode behavior, giving clear usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_batchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses scanning logic, sorting by duration, sequential background execution, post-completion validation, self-advancing behavior, return of a batch ID, and privacy-mode confirmation. This is exceptionally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of five sentences, front-loaded with the primary purpose. Every sentence adds valuable detail—scanning, sorting, validation, self-advancing, and privacy—while remaining efficient. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex batch tool with no output schema, the description covers all essential operational aspects: what it does, how it processes files, validation feedback, automatic advancement, return value, and privacy behavior. It even points to the companion check_batch_progress tool. This is complete for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, giving a baseline of 3. The description adds meaningful context for privacy_mode, explaining the one-time confirmation and that no transcript text is returned. It also clarifies that output_format applies to all files in the batch, which is absent from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb+resource: 'Start an automated sequential batch transcription of all untranscribed files in a folder.' It clearly distinguishes itself from sibling tools by detailing the batch scanning, sorting, and background processing behavior. It also references check_batch_progress, reinforcing its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use when you need to transcribe all untranscribed files in a folder sequentially as background jobs. It does not explicitly state when not to use it or mention alternative tools, but the context is strong enough to guide the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whisper_serverA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses critical behavioral traits: the model stays resident in VRAM for the server's lifetime, the server is bound to localhost only, and certain operations are refused while running to avoid GPU contention. This is transparent about resource consumption and operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, and every sentence adds necessary context (performance benefit, VRAM warning, refusal list, localhost binding). It is rich but not bloated, and is well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a server lifecycle tool of this complexity. It covers the action, why to use it, the GPU resource implication, restrictions on concurrent operations, and the network binding. No output schema exists, but the status action details are in the schema, and the description fills remaining gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with detailed descriptions for each action enum value. The description adds no additional parameter meaning beyond the schema; it focuses on the server's behavior and side effects. Per the high schema coverage baseline, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Start, stop, or check the persistent whisper model server.' It uses a specific verb and resource, distinguishing it from siblings like transcribe_audio and switch_model.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided on when to use the server: it eliminates per-file model-load cost, making it a speedup for many short files. It also tells when NOT to use it: 'stop the server to use' the refused operations like start_batch and generate_subtitles, and warns to start deliberately and stop to free GPU.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.