Server Quality Checklist

Tool Scores

• popcorn_readA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions optional filtering but does not disclose return format, error behaviors, size limits, or whether the tool is read-only. 'Read' implies non-mutating, but details are lacking.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action and includes optional details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite schema coverage, the description omits return value expectations, error handling, and any specifics about how filtering affects output. With no output schema and no annotations, the agent is left guessing about the tool's full behavior, making it incomplete for a 6-parameter read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, providing baseline 3. The description adds conceptual grouping by mentioning 'line slicing' (startLine/endLine) and 'transcript time filtering' (startSec/endSec), helping agents understand the parameters' joint purpose beyond their individual schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Read a file from an analysis bundle' with specific optional behaviors ('line slicing or transcript time filtering'). It differentiates from siblings by focusing on reading files, while siblings suggest, presets, analyze, or backend management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reading files from an analysis bundle but provides no explicit guidance on when to choose this tool over alternatives or when not to use it. The optional filtering behaviors are mentioned but not elaborated with scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• popcorn_analyzeA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the primary actions (extract keyframes and transcripts) but does not disclose side effects, resource usage, output location, potential failures, or how the analysis bundle is returned or stored. The description is too thin for a tool with 16 parameters and no annotation context, leaving significant behavioral unknowns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the primary purpose, and includes a useful recommendation without any fluff. Every sentence contributes meaning, making it appropriately concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (16 params, no output schema, no annotations), the description is minimal but not entirely inadequate. It states the core outputs and suggests a workflow with popcorn_suggest, but it does not explain what the analysis bundle contains or how to read results (e.g., via popcorn_read). The schema covers parameters richly, but the overall context for using this tool in the sibling workflow is incomplete, earning a middle score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers all 16 parameters with 100% description coverage, so the baseline is 3. The description adds value by indicating that 'videoType and objective to use smart presets'—a cross-parameter relationship not explicit in the individual schema descriptions. This enhances the agent's understanding of how to combine parameters for optimal use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Analyze a video file.' and specifies its key outputs: 'Extracts keyframes and transcripts.' This is a specific verb+resource combination that differentiates it from the sibling tool popcorn_suggest, which recommends settings rather than performing analysis. The purpose is immediately clear and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'RECOMMENDED: First use popcorn_suggest to get optimal settings, or specify videoType and objective to use smart presets.' This tells the agent when to use this tool relative to popcorn_suggest and offers an alternative path using parameters. It lacks an explicit 'when not to use' exclusion, which prevents a perfect score, but the guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• popcorn_suggestA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that it returns 'video metadata, suggested video type, and recommended presets' and implies read-only behavior via the verb 'probe'. With no annotations provided, it does not explicitly state that the file is never modified, nor does it mention any permissions or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two compact sentences, placing the core action in the first sentence and usage guidance in the second. Every word adds value, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description covers what it does, what it returns, and when to use it. It also connects to the sibling tool for the next step, making it sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the sole 'path' parameter with a description, giving 100% schema coverage. The description adds only minimal context by calling it a 'video file', which doesn't significantly enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'probe' with the resource 'a video file' and explicitly names the output ('suggested analysis settings'). It also distinguishes itself from the sibling tool popcorn_analyze by stating 'before running popcorn_analyze', making its purpose clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states 'Use this FIRST' and 'before running popcorn_analyze', which provides clear temporal guidance. However, it does not mention scenarios where the tool should not be used or alternative tools for different situations beyond the implied next step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• popcorn_backendsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses read-only behavior through 'List' and 'Shows,' and the phrase 'installed and ready to use' indicates it checks availability. This is adequate for a simple listing tool, although it doesn't detail return format or potential caveats like configuration requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences: the first states the core action, the second explains what it shows and offers a use case. Every word earns its place, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description is remarkably complete. It covers purpose, behavioral transparency, and usage context, making it sufficient for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema defines zero parameters, so there is nothing to add. The baseline for no parameters is 4, and the description correctly does not mention parameters that do not exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List available transcription backends on this system' with a specific verb and resource. It further specifies the output ('Shows which backends are installed and ready to use'), and it distinguishes itself from sibling tools like suggest, presets, analyze, and read, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context with 'Use this to help users choose the best transcription option,' indicating when to invoke the tool. However, it does not explicitly mention alternatives or when not to use it, so it lacks direct exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• popcorn_presetsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden. It discloses a read-only listing behavior ('List all available') which is transparent for a zero-parameter tool. No side effects or edge cases are noted, but the operation is straightforward.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, and the purpose is front-loaded. Excellent conciseness and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with no parameters and no output schema, the description fully covers what it does and why to use it, referencing the related tool popcorn_analyze. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline score is 4. No parameter information is needed in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all available video type presets and objectives' with a specific verb and resource, and explicitly references popcorn_analyze, distinguishing it from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this to understand what options are available for popcorn_analyze.' It doesn't mention alternatives or exclusions, but the directive is clear for the intended workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.