Server Quality Checklist

Tool Scores

• list_watchlistsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, covering the safety profile. The description adds value by specifying the output (watchlists and new-video counts) and the 'Free' qualification, which goes beyond the schema. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence using an em-dash to add detail, with the key resource front-loaded. Every word contributes meaning, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list tool with no parameters and no output schema, the description adequately explains the output (watchlists and new-video counts) and scope (account's Radar watchlists). The 'Free' note is slightly ambiguous but does not detract from overall completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so there is no parameter semantics to explain. The baseline for no-parameter tools is 4, and the description appropriately avoids adding unnecessary parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (Radar watchlists) and the return metric (new video counts), making it distinct from sibling tools like list_channel_videos. It uses a specific noun phrase that effectively conveys the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context about what the tool does but does not explicitly state when to use it over alternatives. It implies its role by describing the watchlists, but lacks explicit guidance on choosing it versus sibling tools like watchlist_activity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watchlist_activityA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations indicate readOnlyHint=true, and the description does not contradict that. It mentions ordering (newest first) and cost (Free), but does not detail side effects or output behavior. However, the read-only nature is adequately covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, conveying the core functionality in two short sentences. It avoids unnecessary detail while remaining informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough context for a straightforward retrieval tool, including the pairing advice for question-answering. It does not describe output schema, but that is not required given the simplicity and the lack of output schema definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The watchlistId parameter has a clear description referencing list_watchlists, which is helpful. The limit parameter lacks a description but is self-explanatory as a numeric limit, so semantics are sufficiently clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving the most recent videos recorded for a specific watchlist, ordered newest first. It also distinguishes itself from siblings by specifying 'one watchlist' and including the pairing hint with search_transcript.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage hint by suggesting pairing with search_transcript, but does not explicitly state when to use this tool versus alternatives like list_channel_videos or get_transcript. It implies usage but lacks direct guidance on selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_channel_videosA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and destructiveHint annotations, the description adds meaningful behavioral context: the operation is free, does not spend credits, requires a Starter plan, and only returns recent uploads. This helps an agent understand cost and entitlement implications without repeating annotation data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences state the core action, accepted input forms, cost behavior, and plan requirement. Every sentence carries useful information with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter, read-only tool with good annotations and a clear description, the content fully covers purpose, input, cost, and access restrictions. No output schema exists, but for a list operation this is acceptable; the description is complete for tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents the single parameter ('Channel @handle, URL, or UC… id'), and the description repeats that same input guidance. Since schema coverage is 100%, the description adds no significant additional semantic value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Resolve') with a clear resource ('a YouTube channel handle, URL or id') and result ('recent uploads'). It clearly distinguishes this from sibling tools that handle transcripts, watchlists, or analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states when to use the tool: to resolve a channel handle/URL/id to recent uploads, and adds access context ('Requires the Starter plan or higher'). It does not explicitly name alternatives or when-not-to-use it, but the context is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_videoA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds significant behavioral details beyond annotations: citations are verified, and if nothing survives it 'says the evidence is insufficient rather than guessing', plus it consumes a quota. The annotations only indicate non-read-only and non-idempotent, so the description adds critical information about evidence verification and fallback behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences that front-load the core purpose and verification logic, then add the cost. Every clause earns its place; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, prerequisites, cost, and fallback behavior, which is quite complete for a query tool. However, it doesn't detail return format or what happens on invalid input, though the lack of an output schema makes some of this less critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers analysisId with a description, but question has no description. The tool description does not elaborate on the question parameter beyond being a question. With 50% schema coverage, the description adds minimal value for parameters, though it implies the question is a natural language query.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Ask a question against a finished analysis' and explains the unique behavior of verifying citations against stored evidence. This distinguishes it from siblings like get_analysis or search_transcript.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It specifies that the analysis must be 'finished', implying a prerequisite, and mentions the cost ('Spends one Watch question from the plan'). However, it doesn't explicitly contrast with alternatives like get_analysis or search_transcript, though the context makes it clear this is for post-analysis Q&A.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcriptA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and idempotentHint=false, suggesting side effects (credit consumption), but the description adds explicit cost per video and warns about large payload sizes. It does not contradict annotations, and adds context beyond them, though it could mention that the full transcript is returned in one response or potential truncation. Still, the added cost and size warnings give useful behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, followed by cost and a usage guideline. No unnecessary words or repetition; every sentence earns its place. It achieves high information density in minimal space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers the essential aspects: purpose, cost, and when to use an alternative. It does not describe error conditions or return formatting, but with no output schema and clear schema descriptions, this is acceptable. Sibling tool context is referenced. A slightly higher score would require mention of response format or edge cases, but the existing content is sufficient for most scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters ('lang' and 'videos'), covering their types, defaults, and constraints (max 25 items). Since schema coverage is 100%, the description adds marginal value—it only restates that one or more videos are accepted and notes 'complete transcript' as output, but does not explain parameter formats beyond the schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches complete transcript text for one or more YouTube videos, using the specific verb 'Fetch' and resource 'transcript text'. It distinguishes itself from the sibling tool search_transcript by explicitly mentioning when to prefer that alternative, so the purpose is unambiguous and well-scoped.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: it names the alternative tool (search_transcript) and explains when NOT to use this tool (when only topic-specific parts are needed), including a rationale (full transcripts are large and mostly irrelevant). It also notes the cost (1 credit per video), helping the agent decide based on resource trade-offs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_videoA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description discloses that this operation spends Watch minutes, returns immediately with an analysisId, completes asynchronously, and reuses prior results if the video was analyzed before. These are essential behavioral details that the annotations do not cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact but information-dense: it covers purpose, visual evidence value, cost, asynchronous polling, and caching in roughly five sentences. Every sentence adds meaningful guidance, and there is no redundant restatement of annotations or schema names.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description still tells the agent what the analysis will contain and how to coordinate with get_analysis for the final result. The async handoff, cost, and cached-result behavior make the full workflow clear and actionable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all three parameters completely, including mode values, defaults, and the meaning of partial and video. The description does not add new parameter-specific details, though it gives helpful high-level context about the analysis result and cost behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: "Start a deep visual analysis of a YouTube video" and lists concrete outputs such as chapters, key moments, on-screen text, and timestamped evidence. It distinguishes itself from sibling tools like search_transcript and get_transcript by stressing that it reads the picture, not just captions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states when to use this tool: when the transcript misses visual content like slides, charts, or demos. It also explicitly tells the agent to poll get_analysis rather than waiting, and notes that previously analyzed videos return ready immediately. It does not mention at all the exact alternative transcript tools other than implicating them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_analysisA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the description earns full credit for adding the async state machine: statuses 'queued,' 'processing,' and 'ready' control whether the analysis field is absent or present, plus the payload contents when ready. No contradiction with annotations; the title and description are consistent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, information-dense sentences: purpose, async status behavior, and payload contents. Every clause earns its place without filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an async fetch operation with a single param, the description fully covers the lifecycle including the 'Free' cost qualifier, the polling conditions, and the result shape (summary, chapters, key points, timestamped evidence). With no output schema present, the description adequately completes the return contract.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and single-param, so the baseline 3 applies without any downside. The description reiterates the analyze_video relationship but adds no format, range, or default guidance beyond what the schema's own note already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses a specific verb ('Fetch') + resource ('analysis') and anchors it to 'started by analyze_video,' which clearly distinguishes it from sibling analyze_video/ask_video. The title 'Read a finished video analysis' further disambiguates its read-only role without restating the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the polling workflow ('poll again... when ready'), establishing when this tool is appropriate in the analysis lifecycle relative to the create operation. It does not explicitly name alternatives with when-not-to-use exclusions, though 'started by analyze_video' hints at the create/read pairing with its sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_transcriptA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already communicate readOnlyHint=false and idempotentHint=false, but the description adds critical context: the tool costs 1 credit per video and that the transcript is fetched server-side to perform the search. Disclosing that this is a paid call addresses a primacy-of-information concern (cost before running). Could go higher with rate limits or error behavior, but the credit disclosure is meaningful and not derivable from annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three focused sentences. The first states the action and output, the second discloses cost and mechanism, the third provides a routing rule. Every sentence earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 100% schema documentation and meaningful annotations (readOnly=false, openWorld=true), the description is nearly complete. It covers the what, the cost, the routing, and hints at the output structure ('matching moments with timestamps and deep links'). Missing details like pagination or error handling are minor; the credit disclosure addresses the most operationally important non-obvious behavior. Returns are probably JSON, but the description's mention of 'cite' implies a usable structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and all parameters are documented in the JSON schema itself, so the description isn't required to repeat them. The description adds value by implying how 'query' relates to matching ('All terms must appear near each other') which is already in the schema, but the description's mention of 'phrase or topic' clarifies the semantic matching. The contextSegments parameter has a default and type in schema, so the 4 is justified because the schema carries the load and the description adds the credit-scope nuance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') with a clear resource ('one YouTube video') and outcome ('return the matching moments with timestamps and deep links you can cite'). It distinguishes itself from the sibling get_transcript by stating 'Use this instead of get_transcript when the user asks what a video says about something.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool ('when the user asks what a video says about something') and names the alternative ('use instead of get_transcript'). It also provides a usage caveat about credits, which helps an agent decide when the cost is justified, going beyond a mere alternative mention.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• accountA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and the description reinforces this by stating it is 'Free' and focused on credit balance. It additionally explains the practical benefit (avoiding failures), which goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, using two sentences to convey purpose and usage. No redundant or irrelevant information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description sufficiently explains what the tool returns (plan and credit balance) and provides actionable context about when to invoke it. It is complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so schema coverage is 100%. The description adds no parameter-specific details, but there is nothing to explain. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving the plan and credit balance for the calling token. It distinguishes from sibling tools (search, transcripts, videos, etc.) by focusing on account information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises checking the account before a large batch to avoid unexpected costs, giving a clear when-to-use directive. Although it doesn't mention alternatives, the tool's unique purpose makes this unnecessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.