Server Quality Checklist

Tool Scores

• analyze_urlA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It does transparently mention input requirements (public/signed https, up to 60 MB) and cost/output parity with analyze_track. Yet it omits failure modes, authentication needs, or what the returned analysis looks like, leaving notable gaps for autonomous use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two compact sentences with no redundant fluff. The key constraint (URL type, size) is front-loaded, and the 'Same output and credit cost as analyze_track' clause efficiently conveys expected behavior without repeating schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, but with no output schema and no annotations, the description must explain what the agent should expect. It gives input constraints and cost/output parity, but not the result structure or error behavior. This is adequate for basic selection but incomplete for nuanced autonomous invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, and the description only adds meaningful context to audio_url (format, size, signed-links). The model parameter's enum values (core/ultra) are left unexplained, and track_name has no description or semantic hint. The description does not compensate for the low coverage of the other two parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: analyzing audio from a URL. It specifies the input type (public or signed https link), size limit, and distinguishes itself from the sibling analyze_track by referencing it for output/cost comparison. This is a specific verb+resource with clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when an agent has an audio URL to analyze, and it notes format and size constraints. However, it does not explicitly state when to prefer this over analyze_track or when not to use it. The reference to analyze_track is indirect and leaves usage decision-making partially open.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_creditsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds the useful behavioral trait that the operation is 'Free' (does not consume credits) and implies a read-only balance check. However, with no annotations provided, the description carries the full burden for behavioral disclosure, and it does not mention response format, error conditions, or authentication requirements. This is acceptable for a simple getter but leaves some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the purpose, the second adds key gratuitous information ('Free'). There is no fluff, and it is front-loaded with the verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is complete enough. It specifies what is returned ('remaining credit balance and API limits') and adds the 'Free' context. Sibling tools are clearly different, so there is no confusion about the tool's role. No further context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is empty and schema coverage is 100%. Per the rubric, a baseline of 4 applies when there are no parameters. The description does not need to explain parameter semantics and does not add anything beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource: the user's remaining TrackTag credit balance and API limits. It also adds 'Free' to signal that this operation does not consume credits, which distinguishes it from sibling tools like analyze_track or analyze_url that presumably consume credits.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, typical scenarios, or exclusions. The 'Free' note implies it could be used to check limits before other operations, but this is not stated explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_jobsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds useful behavioral context: 'Free' (no cost), 'recent' (implies time-ordered), and 'the user's' (scoped to the authenticated user). While it doesn't discuss authentication requirements or pagination details, it gives more than a bare-bones listing and mentions return fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the action and resource, then lists the key return fields and cost. No wasted words; every token earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and no output schema, the description provides a solid overview: what it lists, the fields returned, and that it's free. It doesn't mention ordering or pagination, but the 'recent' qualifier and the limit parameter in the schema partially cover that. Missing authentication/runtime details are less critical for such a straightforward read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (limit) with no description, and schema description coverage is 0%. The tool description does not mention the limit parameter at all, so the agent must infer its meaning from the parameter name. This is a significant gap, even though the parameter is simple and optional.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb ('List') and a clear resource ('the user's recent TrackTag API analyses'), and it enumerates the returned fields (id, status, track, credits). This clearly distinguishes it from siblings like analyze_track (creates analyses) and get_credits (just shows credit balance).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you need to see past analyses, use this tool. However, it doesn't explicitly state when to use it versus alternatives, nor does it give exclusions or edge cases. The sibling list provides some context, but the description itself offers no direct comparison or 'use this when' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_audio_filesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the search directories and partial-name matching, but does not describe the return format (e.g., list of paths), behavior with zero results, or any potential side effects. The mention of 'Free' adds minimal value. Some behavioral context is present, but significant gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and contains no redundant information. Every sentence contributes to understanding the tool's function and use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description provides purpose, scope, search directories, and a concrete use case. It does not explicitly describe the return value, but the phrase 'resolve a file path' implies the output. Given the low complexity and no output schema, this is nearly complete, though a note about multiple matches or no results would elevate it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the single parameter 'query' with a clear example. The description reinforces that it is a partial name but does not add additional semantic value beyond the schema. With 100% schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds audio files on the machine by partial name, searches specific directories, and is distinct from sibling tools like analyze_track or list_recent_jobs. The verb 'Find' and resource 'audio files' make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear when-to-use scenario: resolving a file path when a user mentions a track by name, such as after dragging a file into chat. It lacks explicit exclusions or comparisons to alternatives, but the context is sufficient for basic selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_trackA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description compensates by disclosing credit costs (1 or 2), auto-refund on failure, and file size constraints. It stops short of describing the response format or whether it creates job records, but the disclosed behavior is sufficient for basic agentic decisions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences: first states action and output, second covers cost/failure, third gives usage routing. No repetitive or extraneous information; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of output schema and annotations, the description fully covers the tool's purpose, constraints, costs, and relationships to sibling tools. It even addresses the common ambiguity of whether the agent has a file path vs. a name, making it complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds crucial context for file_path (needs real path, not a name) and reiterates the cost difference for model, even though the schema also says this. This extra guidance improves parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Analyze') and resource ('LOCAL audio file'), lists supported formats and size limit, and states concrete outputs (BPM, key, genres, etc.). It clearly distinguishes from the sibling analyze_url by explicitly noting it is for local files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use and when-not-to-use context: 'if you only know the track's name... call find_audio_files first' and 'For files over 15 MB or already-hosted audio, use analyze_url.' This directly guides tool selection against siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.