Server Quality Checklist

Tool Scores

• get_video_structureA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description effectively discloses cost (1 YouTube API quota unit), transcript fetching details (outside API, zero quota, may be unavailable), and potential slowness. However, it does not mention error handling, rate limits, or response format beyond listing returned fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences that front-load the purpose and then add critical cost/availability context. Every sentence is meaningful with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema and three optional parameters, the description adequately outlines the returned fields (duration, chapters, tags, transcript) and key constraints. It could be more explicit about the response structure, but the current text is sufficient for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with descriptions, so the tool description adds minimal value for parameters. It does not elaborate on the video input format or the effects of includeTranscript/maxTranscriptChars, but the schema provides adequate defaults and types. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches a video's structure (duration, chapters, tags, transcript) for replication. It's specific to outlier videos and the 'Icon Method,' but the jargon may confuse some agents. It distinguishes from sibling tools like find_outliers or get_comment_signal by focusing on structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lacks explicit guidance on when to use this tool versus alternatives. It mentions 'Icon Method verification step' but does not explain when not to use it or provide comparisons with siblings like find_outliers or search_niche_sweep.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_comment_signalA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that comments are relevance-ordered, metrics are simple (questions and demand phrasing), and costs 1 quota unit. It does not explicitly state whether the tool is read-only or any authentication needs, but these are implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a cost note, highly concise and front-loaded. Every sentence adds value: purpose, use case, and cost.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters, no output schema, and no annotations, the description explains the tool's output (top comments and demand metrics) and purpose (verification step). It could describe the return structure more explicitly but is adequate for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal new meaning beyond the schema: it repeats that 'video' can be ID or URL and mentions default values for maxComments. No additional semantic depth.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch' and the resource 'video's top comments plus simple demand metrics'. It distinguishes itself from sibling tools like find_outliers, get_video_structure, and search_niche_sweep by focusing on comment-based demand signal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a specific use case: 'Icon Method verification step: comments prove the topic has unmet demand, not just views.' It also notes the cost in API quota. However, it does not explicitly state when not to use this tool or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_niche_sweepA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains the mechanism (substituting niche into template, running outlier search per niche) and discloses the expensive nature. However, it does not mention if the tool is read-only or any side effects, and output format is unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two clear paragraphs: one for purpose and one for cost. It front-loads the action but could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters and no output schema, the description omits what the returned data looks like. It says 'rank the hits across all niches' but does not specify fields or format, leaving a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds an example ('beginner mistakes {niche}') but does not provide additional meaning beyond the well-documented schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: running an outlier search per niche by substituting a phrase template, ranking hits across niches. It answers a specific question and distinguishes from siblings like find_outliers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description warns about the high cost (percent of daily quota), implying it should be used sparingly and for cross-niche comparison. However, it doesn't explicitly state when not to use it or direct users to an alternative for single-niche searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_outliersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It reveals the tool's cost (~110-130 API units), the logic behind defaults, and that it searches by topic. It does not mention output format or auth requirements, but provides sufficient transparency for typical agent use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, each with a distinct purpose: purpose + logic, defaults, and cost. It is front-loaded with the most critical information and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's purpose and usage well, but lacks details about the output format (e.g., fields returned) and potential pagination or sorting. Given the complexity and no output schema, this is a minor gap but still leaves the agent partially uninformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter is already documented. The description adds little beyond the schema beyond explaining the overall logic and default values (Icon Method). This meets the baseline but does not substantially enrich parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: search YouTube for outlier videos on small channels that outperform expectations. It uses specific verbs ('Search') and resources ('YouTube videos'), and the logic of format-driven vs. audience-driven success distinguishes it from sibling tools like get_comment_signal or get_video_structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (to find replicable format-driven outliers) and provides clear context (Icon Method criteria). However, it does not explicitly state when not to use it or compare with sibling tools, leaving the agent to infer from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.