Server Quality Checklist

Tool Scores

• rouletteB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=false. The description adds 'randomized' — which aligns with non-idempotence — but does not disclose what the tool returns, how randomization works, or any side effects. It offers some extra but limited behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence — compact and front-loaded with the core action. However, the wording is somewhat poetic ('charts recycle forever') which, while memorable, adds little operational clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 9 parameters, no output schema, and 0% parameter documentation, the description is severely incomplete. An agent cannot reliably know what inputs are meaningful or what result format to expect, making successful invocation unlikely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only mentions 'topic' as a filter, leaving 8 other parameters (seed, count, genre, intent, country, exclude, min_episodes, allow_explicit) entirely unexplained. This is far from adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Discover') and resource ('podcasts'), and clarifies the niche ('long-tail', 'randomized probes'). It distinguishes itself from siblings like 'trending' by emphasizing non-chart discovery, though it doesn't name any sibling explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Discover long-tail podcasts by topic' implies a use case for obscure discovery, but there is no explicit when-to-use vs. alternatives, no exclusions, and no pointer to sibling tools like 'trending' or 'peek'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• peekA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, covering safety and external data. The description adds value by specifying the exact output focus (cadence, episode length, latest episodes) and the 'any feed' scope, which matches the openWorldHint. No contradictions, and no adverse behavior is omitted that would be expected for a read-only fetch.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the action and scope, then states the output. No wasted words; every phrase contributes to understanding the tool's purpose and result.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With two parameters and no output schema, the description leaves the 'episodes' parameter unmentioned and does not specify how many episodes are included in 'latest episodes' or the exact return structure. The openWorldHint suggests external content but no error handling or edge cases are addressed. An agent would not know how to correctly set 'episodes' without further inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description indirectly clarifies feed_url via 'any podcast's RSS feed', but it does not explain the 'episodes' parameter at all. The agent has to infer that 'episodes' likely controls how many latest episodes to return, and the default of 5 is not mentioned. With zero schema coverage, the description must compensate and fails to do so for the optional parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and resource ('any podcast's RSS feed'), clearly stating what is returned (cadence, typical episode length, latest episodes). It distinguishes itself from sibling tools like roulette, trending, and listing tools, all of which imply different actions or targets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool to fetch a podcast feed and get summary stats. It does not explicitly name alternatives or exclusions, but the purpose is distinct enough from the siblings that an agent would not confuse it. There is no explicit 'when not to use' guidance, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trendingA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, and idempotentHint=true, covering safety and idempotency. The description adds valuable behavioral context by specifying the output is enriched with feed URLs and episode counts, and that the result can be filtered by genre. This goes beyond the annotations and informs the agent of the expected content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose, the two modes, and the enriched output. There is no wasted text; every phrase adds meaning and the structure is optimal for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple (3 optional params, no required fields) and the annotations cover safety. However, since there is no output schema and the parameter semantics are not fully explained, the description leaves an agent uncertain about how to set limit and country meaningfully. The enrichment detail is helpful, but the return structure and parameter usage are only partially covered, making it minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 3 optional parameters with no description coverage (0%). The description names the 'genre' concept ('overall or by genre') but does not explain what values genre accepts, what limit and country mean, or how they affect results. Given the low schema coverage, the description fails to compensate adequately for the callers' understanding of the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (list/trending), the resource (Apple's top podcasts), and the scope (overall or by genre). It also mentions the enrichment with feed URLs and episode counts, distinguishing its output from a simple ranking. There is no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage context: it lists top podcasts, which is distinct from the sibling tools like roulette (random selection) or peek (inspection). It explicitly tells the agent there are two modes (overall or by genre), but does not explicitly say when not to use it or mention alternatives. Still, the context is clear enough for an agent to route appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_genresA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, covering the safety profile. The description adds the relationship to roulette/trending, which is useful context, but does not add details about the return format, ordering, or any side effects. With annotations handling the core behavioral guarantees, a mid score is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded and contains no filler. Every word contributes to the meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and annotations cover its safe/read-only nature, the description adequately indicates what it returns (genre names) and its relationship to sibling tools. It could be slightly more explicit about the output format (e.g., a list of strings), but the current wording is sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so per the rubric the baseline is 4. No additional semantic explanation is needed since the tool takes no arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists genre names and specifies they are the ones accepted by roulette/trending, distinguishing it from sibling tools. The verb 'list' and resource 'genres' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'accepted by roulette/trending' implies this tool is used when you need genre names for those specific tools. It provides clear context for when to call it, though it does not explicitly mention when not to use it or name alternatives beyond the implicit sibling relation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skill_readA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint, covering safety and repeatability. The description adds the source dependency (skills_list) but does not disclose other behavioral traits like output length or error handling specifics. The mention of 'guidance on error recovery and effective use' is vague and does not add concrete behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly written sentence that front-loads the action and source. The dash adds a note about guidance without bloat. Every word earns its place, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter, no output schema, and strong annotations, the description suffices. It tells the agent what it does and where the input comes from. It does not detail error cases or return format, but these are minor given the tool's simplicity and the mention of 'guidance on error recovery' embedded in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description carries the entire burden. It explicitly states that the 'name' parameter is the skill name and that it should come from skills_list. This adds meaningful meaning beyond the schema's bare 'name' field, though it leaves format specifics (e.g., case sensitivity) unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action (Fetch the full content), the resource (one skill), and the source (skills_list). It distinguishes itself from sibling tools like skills_list (which lists) and roulette/trending (which are unrelated). The title reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'by name (from skills_list)' provides clear context that the name should come from the skills_list tool, implying a workflow of listing first then reading. However, it does not explicitly state when not to use this tool or name alternatives; it only implies the dependency on skills_list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• skills_listA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, covering the safety profile. The description adds meaningful context by defining what skills are (guidance playbooks) and their purpose (using the server well). It does not contradict annotations and provides useful domain context beyond the structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the core action ('List available skills'), immediately defines the resource, and ends with an actionable pointer to the sibling tool. Every word earns its place; there is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless, read-only listing tool, the description is complete. It tells the agent what the tool returns (a list of skills), what those skills are, and how to proceed (skill_read). No output schema exists, but the simplicity makes that acceptable. The annotations cover safety, so nothing is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to document. Per the rubric, a baseline of 4 applies for 0 params. The description adds no parameter information because none exists, and the schema correctly shows an empty properties object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a resource ('available skills'), clarifies they are 'guidance playbooks', and directly distinguishes from the sibling tool skill_read by pointing to it as the next step. This is a precise, unambiguous purpose statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to use this tool (to see the list of skills) and when to use the alternative skill_read ('read one with skill_read'), providing clear routing between the two related tools. This matches the standard of the get_calls example.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.