Server Quality Checklist

Tool Scores

• recognize_fileB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions auto-detection of format from extension, which adds behavioral context. However, it does not disclose whether the tool is read-only, error behavior, output format, or permission requirements. Since annotations are missing, the description carries the full burden and falls short.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the purpose. Every sentence adds value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is adequate but incomplete. It explains the basic function but omits important context such as return format, error handling, or usage restrictions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description does not add extra meaning beyond what the schema already provides for the two parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool recognizes speech from a local audio file, with auto-detection of format from extension. However, it does not explicitly differentiate from the sibling tool 'recognize_speech', which could cause ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'recognize_speech'. The description lacks context on prerequisites, limitations, or appropriate use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• synthesize_speechB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only discloses that output is Base64-encoded audio. Missing details like synchronous/asynchronous behavior, size limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no extraneous words. Front-loaded with the core purpose, followed by input/output format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple tool but missing useful context like supported languages, max text length, or performance traits. Sibling tools are similar in domain, increasing need for clarity on distinct use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so the description adds little beyond the schema. The only extra context is that it returns Base64 audio, but this does not enhance parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'synthesize' and resource 'speech', mentions the service 'SaluteSpeech', and distinguishes from sibling tools like recognize_speech and recognize_file by its text-to-speech nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives (e.g., recognize_speech for speech-to-text), nor does it mention prerequisites or scenario-specific advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recognize_speechB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It only states basic input/output but omits details like audio length limits, required permissions, latency, error handling, or whether the operation is destructive. This is insufficient for a speech recognition tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no unnecessary words. Front-loaded with the core purpose and input/output format. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks details about return format (e.g., response structure), error scenarios, supported audio lengths, or any constraints. For a tool with 3 parameters and no output schema, this is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description adds no extra meaning beyond confirming Base64 audio input. Baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs speech recognition via SaluteSpeech, accepts Base64 audio input, and returns text transcription. It distinguishes from siblings like synthesize_speech (text-to-speech) and recognize_file (likely file-based).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for transcribing audio from Base64 data but does not explicitly state when to use this tool versus alternatives like recognize_file or synthesize_speech. No exclusion criteria or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_task_statusA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It only states 'Check status' with no mention of side effects, error handling, idempotency, or rate limits. This is insufficient for a tool that queries an external async process.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the key verb and resource. Every word serves a purpose with no redundancy or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema. The description adequately conveys the tool's purpose. However, it could be enhanced by mentioning the possible statuses or expected response format to better inform the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with a clear description for 'task_id'. The tool description adds no additional semantics beyond what the schema already provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Check status', the specific resource 'async SaluteSpeech recognition task', and the method 'by ID'. It distinguishes itself from sibling tools like list_models, recognize_file, etc., which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly tells when to use this tool (after obtaining a task ID from an async request), but lacks explicit guidance on when not to use it or alternatives. The usage context is implied but not fully elaborated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description covers the basic behavior (listing models/voices, filtering by type) but, given no annotations, it lacks details on authorization, pagination, or output format, which would be helpful for an agent. It is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with the key action and resource. Every word is necessary, and it is efficiently structured without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with one optional parameter and no output schema, the description is fairly complete. It explains the tool's purpose and filtering ability. Minor improvements could include mentioning that voices are for synthesis and models for recognition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing full info on the 'type' parameter. The description adds no extra meaning beyond the schema, meeting the baseline for a parameter with an enum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and clearly identifies the resource ('available SaluteSpeech models and voices') and purpose ('for recognition and synthesis'), clearly distinguishing it from siblings like 'recognize_file' which perform actual transcription.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing models and voices, but does not explicitly state when to use it versus alternatives like 'get_task_status' or when not to use it. The context is clear but lacks exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.