Server Quality Checklist

Tool Scores

• image_metadataC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses 'no cloud call', which is useful behavioral context. However, it also claims 'requires local file path', which is factually false given the schema allows URLs and base64. Since no annotations exist, the description's inaccuracy is a significant transparency failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is concise, but it packs a false claim about local paths. While the length is appropriate, the inaccuracy prevents it from being fully effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  It lists metadata types and notes offline behavior, but fails to reconcile the local-path statement with the schema's broader input support. It also does not explain when to use this tool versus sibling tools like analyze_image, and since there is no output schema, return details are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage for the 'image' parameter, so baseline is 3. But the description actively adds incorrect information by stating 'local file path required', directly contradicting the schema. This negative contribution reduces the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads image metadata (dimensions, format, color mode, EXIF/GPS) with a specific verb and resource. However, the qualifier 'local' conflicts with the schema's support for URL/base64, slightly muddying the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It mentions 'requires local file path' and 'does not call cloud', implying a context of offline local use, but gives no explicit alternatives or exclusions. The local-path requirement is contradicted by the schema, making the guidance misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_audioA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It reveals a key limitation (URL audio may not be recognized) and the default behavior (transcribe and summarize without a prompt), which adds valuable transparency. It could mention output format or failure modes, but the disclosed traits are useful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core purpose and then adds two key usage notes. No word is wasted, and it is efficiently structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and moderate parameter complexity, the description covers the main purpose, input recommendations, and default behavior. It does not elaborate on output structure or edge cases, but it is sufficiently complete for the tool's simplicity, especially given the rich schema descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds little beyond the schema: the prompt default behavior and audio format recommendations are already present in the schema descriptions. Thus, it does not significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function ('transcribe or summarize audio content') and mentions 'audio understanding/STT', which specifies the resource and action. However, it does not explicitly distinguish itself from the sibling tool 'asr', which likely also handles speech-to-text, so it only partially differentiates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides practical guidance by recommending local files or base64 over URLs, and explains the default behavior when no prompt is given. This gives clear context for use, though it does not explicitly mention when to choose this tool over alternatives like 'asr'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It uses 'query' implying read-only behavior and lists the returned information, but does not explicitly disclose side effects, permissions, or rate limits. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single well-structured sentence that front-loads the purpose and efficiently lists the queried attributes. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-parameter status tool, the description adequately covers what the tool returns (model configuration, current model, limits). It lacks mention of return format or when to use, but given the simplicity and sibling context, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has zero parameters, so schema coverage is trivially 100%. Per rubric, baseline is 4. Description contributes no parameter details because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool queries service status, enumerating specific details (cloud model configuration, current model, limits/processing parameters). This is a distinct purpose from sibling analysis tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for checking system status but provides no explicit guidance on when to use it vs alternatives, nor any exclusions. Siblings are clearly analysis tools, making context implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• asrA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals input format constraints and the effect of the language parameter on accuracy, but it does not describe the output format beyond 'plain text', nor does it mention potential side effects, timeout behavior, or error handling. That leaves gaps for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it starts with the core purpose, followed by use-case context, then constraints, and ends with a performance tip. Every sentence contributes meaningful information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description does not fully specify the return format or structure. It mentions the result is plain text, but not whether it returns a string, JSON object, or includes metadata. It also lacks info on error cases or asynchronous behavior. Given the tool's moderate complexity and missing output schema, the description is decent but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining that specifying the language (auto/zh/en) can improve accuracy, which is not evident from the enum alone. It also reinforces the audio format and size limits already present in the schema, giving practical guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: converting audio to plain text via speech recognition (ASR). It specifies the resource (audio) and the action (transcription), and differentiates from siblings like analyze_audio by focusing on producing text rather than analyzing audio features.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: 'suitable for meeting notes, dialects, noisy environment recordings'. It also gives constraints (wav/mp3, max 10MB). However, it does not explicitly mention when not to use it or name alternative tools such as analyze_audio, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_videoA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses useful behaviors: local videos are auto-compressed, fps affects temporal detail and token consumption. However, it does not explain the return format, whether the operation is read-only, or other behavioral aspects like potential latency or auth requirements. This is a moderate level of disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: two sentences cover the purpose, supported input types, compression behavior, and fps guidance without any fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters and no output schema, the description provides sufficient context: it explains what the tool does, what inputs are accepted, and key usage notes. It lacks an explicit description of the return value, but the purpose ('描述视频内容、按时间顺序总结') implies a textual summary, which is adequate given the schema's detailed parameter descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining the fps parameter's effect on detail and token usage, recommending the default, and noting that local videos are auto-compressed—details not in the schema. This enrichment justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: '视频理解' (video understanding) and describes what it does: '让模型描述视频内容、按时间顺序总结' (let the model describe video content and summarize chronologically). This differentiates it from sibling tools like analyze_image and analyze_audio by its focus on video content understanding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: for video understanding with support for local files, URLs, or base64, and includes guidance on fps (default 2, range 1-30, larger means finer detail but more tokens). It does not explicitly name alternatives or exclusions, but the context is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_imageA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the automatic compression behavior for local images, its effect on OCR vs. original detail, and how to disable it with compress=false. This is valuable behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose. Every clause earns its place: capabilities, image vs. batch, presets vs. prompt, and compression policy. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and no output schema, the description covers the main invocation patterns, task types, and a key behavioral caveat. It doesn't explain return formats, but that is expected when no output schema is provided. The description is sufficient for an agent to select and use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaning by clarifying the difference between image and image_batch, and by explaining the practical implications of the compress parameter (OCR clarity vs. detail/token cost). This goes beyond the schema's basic field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs image understanding and OCR, listing concrete use cases (description, text recognition, chart/UI/error diagnostics). This distinguishes it from sibling tools like analyze_video, analyze_audio, and asr by explicitly focusing on images.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance: single image vs. multi-image (image_batch), preset selection vs. free-form prompt, and compression behavior. It does not explicitly mention exclusions (e.g., 'use analyze_video for video'), but the tool name and sibling list make the domain obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.