Server Quality Checklist

Tool Scores

• batch_analyseB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description fails to disclose behaviors such as read-only vs. destructive, authentication needs, rate limits, or what 'analyse' entails. The description adds minimal context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The main description is a single concise sentence. The additional Args section provides necessary parameter details. It is appropriately sized with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the batch nature and output schema existence, the description could mention handling of multiple URLs, limits, or result aggregation. It is adequate but lacks some context for a complex batch operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining the urls parameter as 'List of YouTube URLs to analyse' and config_file as 'Path to a YAML/JSON file containing a list of URLs'. This adds meaning beyond the schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes multiple YouTube videos from URLs or a config file. It distinguishes itself from sibling tools like analyse_video (single video) and analyse_x_feed (X/Twitter feed).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives (e.g., analyse_video for single videos). The description implies batch usage but does not state exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyse_videoA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It fails to disclose any behavioral traits such as side effects (none expected), authentication requirements, rate limits, or whether it modifies data. The description only mentions outputs without transparency on operational aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two paragraphs plus Args section, front-loaded with key info. Some redundancy in listing URL formats, but overall efficient. Every sentence contributes to understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and that an output schema exists (not shown), the description covers core functionality but lacks details on error handling, video availability issues, or input validation. It is adequate but has gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. The Args section provides brief explanations: 'url: YouTube video URL' and 'custom_prompt: Additional analysis instructions (optional)'. The general description also lists supported URL formats. This adds some value but does not fully compensate, e.g., no detail on what 'additional analysis instructions' entails.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'analyse' and the resource 'a single YouTube video', and lists specific outputs (transcript, key takeaways, TLDR, social hook). It distinguishes from siblings by focusing on single video and YouTube platform.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: when user provides a YouTube URL or asks to analyse a YouTube video. Lists supported URL formats. Does not explicitly state when not to use or mention alternatives, but the context implies single-video use only.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_imageA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the output style (comic-book, bold colours, panel divisions, text labels) and implies it creates a visual summary, but does not discuss potential side effects, authentication, rate limits, or whether the image is saved or returned. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with three short sentences front-loading the purpose and usage context. The Args section is straightforward, though it partially repeats earlier information. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one complex required parameter and no annotations, but an output schema exists. The description does not explain what the output is (e.g., file format, storage location) beyond 'visual summary'. With the output schema available, the description could be more complete by noting the output's nature or additional constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The 'Args' section only states the analysis parameter is a result object from certain tools, without detailing its structure or required fields. The schema allows any additional properties, offering no clarity, leaving the agent to infer the object's format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool generates a comic-book style infographic image from a video analysis or X digest result, clearly identifying the resource and verb. It distinguishes from sibling tools by specifying it should be used after analyse_video or analyse_x_feed, making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool: after analyse_video or analyse_x_feed, and also mentions batch_analyse as an alternative source. While it doesn't explicitly state when not to use it, the context is sufficient for most scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyse_x_feedA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It describes fetching, filtering, and generating digest, but lacks details on authentication, rate limits, or data freshness. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections: purpose, usage hint, and argument details. Slight redundancy in first sentence but overall concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all necessary aspects for a feed analysis tool: what it does, parameters, defaults, usage context. Output schema exists, so return values are not needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description explains each parameter well: usernames (defaults to configured), topics (defaults to configured), hours_back (default 24, suggests 168 for weekly). Adds significant value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it analyzes X/Twitter posts and generates a digest. It distinguishes well from sibling tools (analyse_video, batch_analyse, etc.), which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use ('when user asks about X posts, tweets...'). Does not mention when not to use, but siblings are unrelated, making context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_channel_videosA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains parameters and defaults but omits details like error handling, rate limits, or sorting order of results. Acceptable but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: one line intro, then a clear args list. Every sentence adds value, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so return values need not be explained. Description covers inputs and filter. Could mention that results are sorted by publish date (descending) for full clarity, but minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet description adds full meaning for all three parameters: channel_id, max_results (with default and max), and published_after (ISO date). Adds significant value beyond bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it fetches recent videos from a YouTube channel, including the requirement for a YouTube Data API key. This distinguishes it from sibling tools like 'analyse_video' which perform analysis, not listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Mentions the prerequisite of an API key, which is helpful. No explicit when-not or alternatives, but sibling tools are for different tasks, so usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_to_githubA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It states the tool pushes markdown files and implies git commits via the commit_message parameter, but does not mention authentication requirements, overwrite semantics, or error handling. More detail is needed for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise paragraphs with no fluff. The first paragraph states purpose and usage timing; the second details parameters. Minor redundancy (first two sentences cover similar ground) keeps it from a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 params, dependency on other tool outputs) and the existence of an output schema, the description provides a complete usage flow and parameter explanations. It lacks return value details, but the output schema fills that gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description carries full burden and succeeds. It defines 'analyses' as results from analyse_video/batch_analyse, explains 'image_paths' structure, and ties 'x_digests' to analyse_x_feed. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Push', 'Sync') and explicitly states it stores analysis results to a GitHub repo. It clearly distinguishes itself from sibling tools that analyse or generate content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use after analyse_video, analyse_x_feed, or generate_image to persist results', providing clear when-to-use guidance and naming alternative actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.