Server Quality Checklist

Tool Scores

• get_artistA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes far beyond the annotations. It explains that null/empty fields are the ordinary state, not a failed read. It details how date of birth and nationality are returned (as raw free text). It states the sorting order (by year of release, never by importance). This adds substantial behavioral context that annotations alone could not convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is fairly long but each sentence adds value: purpose, usage dependencies, behavioral caveats, and attribution requirement. It is front-loaded with the core purpose. Minor deduction because the attribution instruction could be shortened, but overall it earns its length with substantive content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is an output schema, the description does not need to detail return format. However, it richly explains what the output represents, edge cases (null fields), data fidelity issues (birth dates as-is), and attribution. The complexity is low (2 params, 1 required), and the description is fully complete for an AI agent to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the input schema already explains both parameters (artist_id with pattern/example, limit with default/max/min). The description reinforces that artist_id comes from specific other tools and explains what limit controls ('Maximum songs of the discography to return'), but does not add new semantic meaning beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool reads (an artist's page on Bide & Musique), listing specific content: names, catalogue notes, and songs with years. It distinguishes itself from siblings by mentioning that it takes the 'artist_id' returned by search_songs and get_song, and the content differs from a song or search result.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool (when you have an artist_id from search_songs or get_song) and what the tool is not (a biography). It also provides a usage directive: 'When you show an artist to a user, credit Bide & Musique and link the page.' This is clear guidance that goes beyond mere purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_songA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, openWorldHint), the description reveals key behaviors: missing fields return null or empty rather than guessed, a counter not printed is unknown (not zero), lyrics inclusion can be controlled via a parameter, and the credit requirement. No contradictions with annotations—all align. This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear front-loaded purpose, followed by parameter guidance, behavioral details, and a final usage note. It contains several sentences, but each adds value—no redundancy. It could be slightly tightened (e.g., the credit sentence is somewhat tangential), but overall it efficiently conveys necessary information for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple fields, null handling, lyrics behavior) and the presence of an output schema for return structure, the description covers all essential aspects: input source, parameter behavior, data completeness semantics, lyrics control, and attribution requirements. Nothing important is missing, making it fully complete for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds significant value beyond the schema: it explains the source of song_id (from search_songs), the rationale for setting include_lyrics false (to avoid large transcripts when metadata is needed), and mentions an output field ('lyrics.available') that informs decision-making. This goes well beyond the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool reads one song's record and lists specific fields (year, writers, composers, duration, etc.), with a clear verb ('Read') and resource ('song's record'). It distinguishes from siblings by specifying it takes a 'song_id' from search_songs, implying a targeted lookup. A definitive 5 for clarity and specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool (to read a specific song's record) and offers parameter-level advice (set include_lyrics to false when the question is about metadata like year or label). It connects to search_songs as the source of the song_id. However, it does not explicitly contrast with siblings like get_random_song or list_new_songs, which would earn a 5. A strong 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_random_songA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint true, destructiveHint false), the description adds significant behavioral context: random draw mechanism over served ids with redraw on invalid id, condition about transcriptions being included, and a requirement to credit Bide & Musique when displaying results. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than strictly necessary, but every sentence earns its place: purpose, usage, draw mechanics, transcription behavior, and attribution requirement. It is well-structured but slightly verbose, so a 4 is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully equips the agent to invoke the tool correctly: it explains the randomness, fallback behavior, transcription inclusion, and attribution duty. The output schema exists, so return values need no further explanation. Complete for a complex random-read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is trivially 100% covered, so the baseline is 4. The description adds no parameter details because none exist, but it does clarify the outcome is the same as get_song, which indirectly helps understand the return shape.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource: 'Read one record drawn at random from the Bide & Musique collection.' It clearly distinguishes itself from siblings by stating 'same answer as get_song, on a record nobody chose' and later 'For a question about a particular song, use get_song.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance: 'Use it to browse the collection. For a question about a particular song, use get_song.' This gives an explicit when-to-use and clearly directs to an alternative for a different use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_new_songsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond annotations by explaining behavior not covered: the fixed-size feed with no second page, lack of sorting (despite newest-first appearance), and how entries combine artist/song into one line. It also tells the agent to parse 'published_at' instead of position. All annotations (readOnlyHint, idempotentHint, openWorldHint) are consistent; no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-front-loaded with the core purpose, but the middle paragraph includes multiple nuanced behavioral notes (one-entry-per-line parsing, order caveats) that could be restructured for faster scanning. Still, every sentence adds value with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 optional param, no nested objects), the description fully covers return shape (id, listed_as, published_at), subtle ordering behavior, and how to use the result. The presence of an output schema reduces burden further. All agent decision points are addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter (limit) already described. The description adds context that limit takes from the 'head of the feed,' confirming it pulls from the start (newest entries). This adds slight value beyond the schema's default/minimum/maximum descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns recently catalogued records from Bide & Musique, contrasting with station playlists. It specifies that the feed has a fixed number of entries, no pagination, and entries are ordered by publication date (newest first). This uniquely distinguishes it from siblings like search_songs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: to see what has been added lately, not what is playing. Provides alternatives: use get_song on the id for full record details. Also warns that count does not reflect total collection size. 'credit Bide & Musique and link the page' gives a when-showing-to-user directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_songsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, openWorldHint, idempotentHint=true, destructiveHint=false. The description adds significant behavioral context beyond annotations: the year axis drops extra words, quoting fails, search uses AND matching inside words, results include song id/artist/page/fields but not matched lyrics, count is total across pages, pagination with 'page' parameter. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence serves a purpose—it covers the collection nature, axis behaviors, search logic, pagination, and attribution. It is front-loaded with the collection context. Minor structural improvement could be made (e.g., bullet-like separation of axis behaviors), but it remains clear and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 axes, special rules for year and quoting, pagination, output format), the description is remarkably complete. It explains what results contain (song id, artist, page, year, label, catalogue, writers), how to get more pages, and the fact that the count is total. The output schema is not provided in the prompt, but the description covers its key fields. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. However, the description adds deep semantic value: explains the meaning and appropriate uses of each search_type value, that query should be in French, how year axis behaves differently, and that quoting a phrase yields no results. This goes well beyond the schema's bare descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool searches the Bide & Musique collection of French songs, with a specific focus on forgotten ones hand-catalogued since 2000. It distinguishes the tool from siblings like get_song (single retrieval) and get_random_song by focusing on search across multiple axes. The verb 'search' and resource 'collection' are explicit and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides extensive guidance: explains that search_type is required, each axis answers a different question, year must be a single four-digit number, how to combine year with other terms, use lyrics to find songs from remembered lines, quoting returns nothing, AND logic for multiple keywords, pagination behavior, and the need to credit the source. Also warns about the count being across all pages, not just the current one. This covers when and how to use the tool effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.