Server Quality Checklist

Tool Scores

• scan_catalogA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses scanning behavior, file types, subfolder inclusion, report contents (counts, confidence), and rate limits. It does not mention if files are modified, but 'audit' implies read-only. Good coverage for a non-destructive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a headline, details, and tier info. It is slightly verbose but each sentence adds value. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description gives a good overview of the return report. It covers scope, limits, and file types. Lacks details on error handling or non-music files, but sufficient for an agent to understand the tool's function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%. The description explains folder_path (absolute path) but does not mention max_files. The phrase 'scans every audio file it finds' contradicts the default max_files=200, potentially misleading the agent. Missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool audits an entire folder of music files for AI-generated tracks, specifies supported file types, and distinguishes it from siblings by emphasizing 'ENTIRE folder' mode. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions pointing at a library folder and daily limits but does not explicitly state when to avoid this tool or when to use sibling tools like detect_ai_music for single files. The context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_ai_mediaA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fails to disclose key behaviors like required permissions, file size limits, or whether the call is read-only. It only states 'Returns the raw SongCheck assessment' without explaining what that means.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words. Purpose, input format, and output are front-loaded, making it easy to scan. Every sentence adds essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 param, no output schema or annotations), the description covers purpose, input, and return type adequately. It lacks details on error handling or output structure but is sufficient for basic invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides no description for the single parameter (0% coverage). Description compensates by clarifying file_path must be absolute, local, and with specific extensions (.jpg/.png/.webp/.mp4/.mov), adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool detects if an IMAGE or VIDEO file is AI-generated. It specifies supported formats (jpg/png/webp for images, mp4/mov for videos) and distinguishes from sibling 'detect_ai_music', leaving no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to pass an absolute path to a local file and lists acceptable formats. While it doesn't mention when not to use (e.g., for audio or remote files), the context is clear enough for most use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• songcheck_healthA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly indicates the tool is a safe, read-only health check with no destructive behavior. The behavior is obvious and fully disclosed, though it could mention that it returns a status indicator.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, extremely concise and front-loaded. Every word is essential; there is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a health check with no parameters or output schema, the description is mostly complete. It could be slightly improved by specifying the response format (e.g., boolean or status), but the current text is sufficient for an agent to understand its purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, giving 100% schema coverage. Per the guidelines, 0 parameters earns a baseline score of 4. The description adds no extra parameter details, which is acceptable since no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') and clearly identifies the resource ('SongCheck detection service') and outcome ('is online'). It distinguishes from sibling tools (detect_ai_media, detect_ai_music, scan_catalog) which focus on detection/scanning rather than service status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for verifying service availability, which is typical for a health check. However, it does not explicitly state when to use this tool vs alternatives or provide any exclusions. For a simple health check, this is adequate but not exemplary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_ai_musicA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully discloses behavioral traits: returns verdict, probability, confidence, provenance signals. Notes tuning limitation for music. Does not mention error handling or non-destructive nature, but covers key behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, input requirements, output and limitation. Every sentence adds value, no redundancy. Front-loaded with primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and one parameter, description covers input, output format, and limitation. Sibling tools provide context. Missing error behavior slightly reduces completeness, but largely sufficient for proper invocations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'file_path' has 0% schema description coverage. Description compensates by specifying absolute path requirement and allowed formats (.mp3, .wav, .m4a, .flac), adding meaningful semantic context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Detect whether an AUDIO file is AI-generated music', specifying the verb (detect) and resource (audio file). Differentiates from siblings like detect_ai_media by noting it is 'tuned for MUSIC'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage: 'Pass an absolute path to a local audio file' with supported formats. Includes exclusion note for synthetic-voice clips without music, guiding when not to use. Does not explicitly name alternatives, but sibling context implies detect_ai_media.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.