Server Quality Checklist

Tool Scores

• op3_show_downloadsA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It describes the output shape (monthly, weekly breakdown, average) but does not disclose whether the operation is read-only, any authentication requirements, rate limits, or data freshness. With no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core functionality and then providing usage guidance. Every sentence adds value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is simple with one parameter and no output schema, the description gives a decent overview of what is returned. However, it lacks detail on the exact format of the 'week-by-week breakdown' or the structure of the summary. For a tool with no output schema, more detail would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the single parameter described in the schema. The description adds value by specifying the UUID comes from op3_get_show, providing context beyond the schema's description of '32 hex chars.'

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves a show's recent download summary including monthly downloads, week-by-week breakdown, and average weekly downloads. It distinguishes from siblings like op3_downloads_timeseries by focusing on a summary rather than a timeseries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use this for 'how many downloads does my show get' questions' and notes the prerequisite of obtaining a show UUID from op3_get_show. It provides clear context but does not specify when not to use it or mention alternatives explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• op3_episode_downloadsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It reveals the read-only nature implicitly and mentions data recency ('recent episodes'), but lacks details on rate limits, data freshness, or whether all episodes are returned. This is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: one defining functionality with clear data details, and one providing usage guidance. Every phrase is informative, with no redundancy or filler. It is appropriately sized and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description compensates by listing the time windows returned (first 1/3/7/30 days and all-time), which is good. However, it lacks details on output format (e.g., episode identifiers, whether titles are included) and any caveats about data availability. Still, it covers the essential aspects for an API of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all three parameters. The description adds minimal value beyond the schema, only restating that 'limit' caps episodes and that 'show_uuid' is needed. Baseline 3 is appropriate as the description does not significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get', the resource 'per-episode download counts for a show from OP3', and specifies the time windows (1/3/7/30 days and all-time). It also distinguishes from siblings by focusing on per-episode data and mentions comparison use case, which sets it apart from aggregate or timeseries tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states a prerequisite ('Needs a show UUID') and provides guidance on using 'limit' to cap results. However, it does not contrast with sibling tools or specify when not to use this tool, which would improve clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• op3_get_showA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Describes what is returned and optional episode listing, but lacks details on error handling, rate limits, or authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences front-loaded with purpose and usage, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Describes return values (UUID, title, GUID, stats page URL) and optional episode inclusion. Lacks output schema, but this is adequate for a simple lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by explaining identifier options (UUID, guid, base64 feed URL), default for include_episodes to keep response small, and episode_limit default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states the tool looks up a podcast show and returns UUID, title, podcast GUID, and stats page URL. Distinguishes from siblings by calling itself the entry point that other tools depend on.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clearly states it's the entry point and most other tools need the returned UUID. No explicit when-not or alternatives, but context strongly implies use this first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• op3_top_appsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return shape (download count and percent share), time range (last three months), and prerequisite (UUID). It does not mention rate limits, auth needs, or other behaviors, which would be helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no fluff. It front-loads the main action and time range, then adds return details and prerequisite. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 params, no output schema), the description is complete. It explains what the tool does, what it returns, the time range, and the required parameter. An agent can use it correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good parameter descriptions. The description adds value by explaining the time range and return shape, which goes beyond the schema. It compensates well for the lack of output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get', the resource 'top podcast apps and players', and the scope 'over the last three calendar months'. It also answers the question 'what apps do my listeners use', distinguishing it from sibling tools like op3_top_countries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the prerequisite 'Needs a show UUID' and implies usage for app breakdown. However, it does not explicitly mention when not to use it or provide alternatives, though the sibling list gives context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• op3_downloads_timeseriesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that records are verbose and the tool is a low-level feed, implying a read-only, data-intensive operation. However, it does not explicitly state whether the tool is read-only or detail authentication requirements, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, with only four sentences that each serve a distinct purpose: stating the tool's function, describing output, placing it among siblings, and providing usage guidance. There is no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description does a good job of conveying the return structure (time, country, app, device) and the need to limit results. However, it does not fully describe the output format or pagination behavior, which would be helpful for complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are described in the schema (100% coverage), but the description adds value by explaining the output structure and verbosity, and by providing context for the limit parameter. This goes beyond the schema's descriptions to help the agent understand parameter impact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool fetches raw download records for a show over a date range, returning individual events (time, country, app, device). It explicitly distinguishes itself as the low-level feed behind other tools, differentiating from sibling tools like op3_show_downloads and op3_top_countries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('when you need to filter by episode or a specific date window') and when to prefer alternatives ('For totals or geography summaries, prefer op3_show_downloads / op3_top_countries'). It also advises to keep the limit low due to verbosity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• op3_top_countriesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it counts raw records and aggregates client-side, uses a sample, defaults to recent data via window_days, and notes rate limits. This is comprehensive and confirms no destructive behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient, with a clear first sentence stating the purpose, followed by essential context about limitations and defaults. Every sentence adds value, though the technical note about OP3's ordering could be slightly more integrated. Still, it avoids fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, no output schema in provided schema), the description explains how the tool works (client-side aggregation from raw records), what each result contains (download count and percent share), and important caveats (representative sample, rate limits). This is complete enough for an agent to understand the tool's behavior and output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 7 parameters are described in the input schema (100% coverage), so baseline is 3. The description adds value by explaining the rationale behind defaults (e.g., window_days to keep sample recent, max_records for speed and rate limits) and the note about OP3's lack of native geography query, which clarifies the tool's design. This elevates it above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and the resource ('top listener countries (or regions) for a show'). It distinguishes itself from sibling tools by focusing on geographic aggregation, unlike op3_downloads_timeseries or op3_episode_downloads which handle time series or per-episode data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (to get top countries/regions) and provides context on its limitations (representative sample, not exact). However, it does not explicitly advise against using it in cases where exact totals are needed or suggest alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.