Server Quality Checklist

Tool Scores

• get_artistA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral detail beyond the annotations: it explains null/empty fields as normal, the exact formatting of date of birth and nationality, the ordering of rows by year, and the requirement to credit Bide & Musique. This fully discloses the tool's behavior and aligns with the readOnlyHint and idempotentHint annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than minimal but every sentence contributes meaningful information: purpose, input source, null behavior, date format, ordering, and citation. It is front-loaded with the purpose and weaves in necessary context without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description does not need to enumerate return fields. It covers all critical behavioral aspects—null handling, field formats, ordering, and attribution—making it complete for the tool's complexity. The explanation of the sparse catalogue helps the agent interpret results appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and both parameters have descriptions. The description adds value by explaining that artist_id is the identifier returned by other search tools, which is not in the schema. However, it does not clarify the limit parameter's behavior relative to the 'every song' claim, so it doesn't fully resolve potential ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads an artist's page and enumerates its contents (names recorded under, catalogue notes, and songs with years). It distinguishes itself from siblings like search_songs and get_song by targeting the artist resource specifically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells the agent that artist_id comes from search_songs, get_song, and get_random_song, providing the necessary context for when to use this tool. It also sets expectations about the sparse nature of artist pages, but does not explicitly state when not to use it or name alternatives for similar queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_random_songA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and destructiveHint=false, and the description adds valuable behavior: the draw covers served ids, invalid ids are redrawn, and results include transcriptions if present. It also adds attribution requirements for showing to users, which is beyond the annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient, front-loading the core purpose, then usage, then behavior, and ends with attribution. Every sentence adds value; the only slight inefficiency is the last sentence about crediting, which could be shorter, but it's necessary context. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one param, read-only, no output schema needed as it's described as same as get_song), and the description covers purpose, usage, behavior, parameter purpose, and attribution. Complete for its complexity. The output schema likely mirrors get_song, so no need to detail returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the single parameter with full description, so the bar for added value is modest. The description adds why the parameter exists (to avoid heavy calls in browsing chains) and what it controls (whether to include transcription), though it doesn't add syntax details. Given full schema coverage, a 4 is justified for the contextual usage guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads a random record from the Bide & Musique collection, same answer as get_song. It distinguishes itself from siblings by explicitly contrasting with get_song for particular song queries, and from search_songs and list_new_songs by its random selection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use: to browse the collection, and when not to: for a question about a particular song, use get_song. This direct alternative guidance is among the best evidence for this dimension.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_songA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, openWorldHint), the description reveals key behaviors: missing fields return null or empty rather than guessed, a counter not printed is unknown (not zero), lyrics inclusion can be controlled via a parameter, and the credit requirement. No contradictions with annotations—all align. This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear front-loaded purpose, followed by parameter guidance, behavioral details, and a final usage note. It contains several sentences, but each adds value—no redundancy. It could be slightly tightened (e.g., the credit sentence is somewhat tangential), but overall it efficiently conveys necessary information for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple fields, null handling, lyrics behavior) and the presence of an output schema for return structure, the description covers all essential aspects: input source, parameter behavior, data completeness semantics, lyrics control, and attribution requirements. Nothing important is missing, making it fully complete for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds significant value beyond the schema: it explains the source of song_id (from search_songs), the rationale for setting include_lyrics false (to avoid large transcripts when metadata is needed), and mentions an output field ('lyrics.available') that informs decision-making. This goes well beyond the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool reads one song's record and lists specific fields (year, writers, composers, duration, etc.), with a clear verb ('Read') and resource ('song's record'). It distinguishes from siblings by specifying it takes a 'song_id' from search_songs, implying a targeted lookup. A definitive 5 for clarity and specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool (to read a specific song's record) and offers parameter-level advice (set include_lyrics to false when the question is about metadata like year or label). It connects to search_songs as the source of the song_id. However, it does not explicitly contrast with siblings like get_random_song or list_new_songs, which would earn a 5. A strong 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_new_songsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond annotations by explaining behavior not covered: the fixed-size feed with no second page, lack of sorting (despite newest-first appearance), and how entries combine artist/song into one line. It also tells the agent to parse 'published_at' instead of position. All annotations (readOnlyHint, idempotentHint, openWorldHint) are consistent; no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-front-loaded with the core purpose, but the middle paragraph includes multiple nuanced behavioral notes (one-entry-per-line parsing, order caveats) that could be restructured for faster scanning. Still, every sentence adds value with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 optional param, no nested objects), the description fully covers return shape (id, listed_as, published_at), subtle ordering behavior, and how to use the result. The presence of an output schema reduces burden further. All agent decision points are addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter (limit) already described. The description adds context that limit takes from the 'head of the feed,' confirming it pulls from the start (newest entries). This adds slight value beyond the schema's default/minimum/maximum descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns recently catalogued records from Bide & Musique, contrasting with station playlists. It specifies that the feed has a fixed number of entries, no pagination, and entries are ordered by publication date (newest first). This uniquely distinguishes it from siblings like search_songs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: to see what has been added lately, not what is playing. Provides alternatives: use get_song on the id for full record details. Also warns that count does not reflect total collection size. 'credit Bide & Musique and link the page' gives a when-showing-to-user directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_songsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, openWorldHint, idempotentHint=true, destructiveHint=false. The description adds significant behavioral context beyond annotations: the year axis drops extra words, quoting fails, search uses AND matching inside words, results include song id/artist/page/fields but not matched lyrics, count is total across pages, pagination with 'page' parameter. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence serves a purpose—it covers the collection nature, axis behaviors, search logic, pagination, and attribution. It is front-loaded with the collection context. Minor structural improvement could be made (e.g., bullet-like separation of axis behaviors), but it remains clear and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 axes, special rules for year and quoting, pagination, output format), the description is remarkably complete. It explains what results contain (song id, artist, page, year, label, catalogue, writers), how to get more pages, and the fact that the count is total. The output schema is not provided in the prompt, but the description covers its key fields. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. However, the description adds deep semantic value: explains the meaning and appropriate uses of each search_type value, that query should be in French, how year axis behaves differently, and that quoting a phrase yields no results. This goes well beyond the schema's bare descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool searches the Bide & Musique collection of French songs, with a specific focus on forgotten ones hand-catalogued since 2000. It distinguishes the tool from siblings like get_song (single retrieval) and get_random_song by focusing on search across multiple axes. The verb 'search' and resource 'collection' are explicit and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides extensive guidance: explains that search_type is required, each axis answers a different question, year must be a single four-digit number, how to combine year with other terms, use lyrics to find songs from remembered lines, quoting returns nothing, AND logic for multiple keywords, pagination behavior, and the need to credit the source. Also warns about the count being across all pages, not just the current one. This covers when and how to use the tool effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.