Server Quality Checklist

Tool Scores

• list_subtitle_tracksA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It implies a read-only operation via 'List,' but does not disclose details about output format, permissions, or any side effects. Since the tool is simple and the output schema exists, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, directly state the purpose and usage context. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with one parameter and an output schema, the description covers purpose and when to use it. It doesn't need to explain return values due to the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'video' is fully described in the schema as 'YouTube URL or 11-character video ID.' The description does not add additional semantic meaning beyond the schema, which already has 100% coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List the subtitle tracks available for a video,' using a specific verb and resource. It distinguishes itself from sibling tools (get_subtitles, search_subtitles) by focusing on listing available tracks rather than retrieving or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Use this when unsure which languages exist, or whether captions are human-written or auto-generated.' This gives clear context, though it doesn't mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_subtitlesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context about the return format (matching passages with timestamps and deep links) and performance characteristics (cheaper than reading the whole transcript). However, it does not mention potential failure modes, language availability, or limits, leaving some behavioral aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary purpose, then the benefit, then a follow-up suggestion. Every sentence earns its place, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (all return values defined elsewhere), 5 parameters well-described in the schema, and clear sibling tools. The description adds the missing contextual piece: when to use it and how to combine it with get_subtitles, making it complete for the agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with rich descriptions, so the baseline is 3. The description does not add meaning beyond the schema, only referencing the workflow indirectly with 'time range' for follow-up, which doesn't clarify any parameter. Thus a 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds where a phrase is said in a video, using a specific verb ('Find') and resource ('where a phrase is said in a video'). It distinguishes itself from siblings by explicitly positioning it as a search step and mentioning follow-up with get_subtitles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool ('Cheaper than reading the whole transcript') and names a specific alternative/follow-up (get_subtitles with a time range). It lacks an explicit 'when not to use' statement, so it doesn't fully meet the 5-level, but it's well-guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_subtitlesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It clearly explains truncation at max_chars, the resume mechanism via start_time, and the fallback from human-written captions to auto-generated tracks. These are meaningful behavioral insights beyond what annotations would provide. It does not mention caching behavior, but that is a minor omission given the schema covers the refresh param.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, with the core purpose in the first sentence and key behavioral details in a short second paragraph. Every sentence earns its place: the truncation/resume mechanism is explained in a clear, actionable way without fluff. It is front-loaded and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and a rich schema, the description covers the core behavior (reading text, fallback, truncation) but does not explain caching/refresh or language fallback in the description itself. However, these are fully documented in the schema, and an output schema exists, so the description need not detail return structure. It is sufficiently complete for an agent to use correctly, though a brief note on caching would make it more self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the relationship between max_chars and start_time for paginated reading, and clarifies the return behavior when truncation occurs. This goes beyond the schema's bare parameter definitions. Other parameters like lang, timestamps, and end_time are not explained in the description but are well-documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific action: 'Read a video's subtitles as text.' It also distinguishes the tool from siblings (list_subtitle_tracks, search_subtitles) by focusing on reading the actual subtitle content rather than listing or searching tracks. The inclusion of fallback behavior further clarifies exactly what is returned.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reading subtitle text, but does not explicitly state when to prefer this tool over the sibling tools. There is no mention of 'use this when you need the full transcript' or 'use list_subtitle_tracks to see available tracks.' It provides clear context but lacks explicit alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.