Server Quality Checklist

Tool Scores

• loom_explainA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the full burden. It transparently discloses the output format (plain text), the content scope (ADR grounding, F-number references, execution-time values), and the return envelope (ok=true with text, ok=false with M-code diagnostics). This covers key behavioral traits, though it stops short of stating side-effects or permission needs, which are likely immaterial for an explain operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient: the first sentence states the core purpose, and the second adds key output characteristics and error behavior. No wasted words or redundant restatements of the tool name. A slight lack of structured details about parameters keeps it from a perfect score, but it remains compact and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a solid overview of what the tool does and what the output contains, and the output schema likely covers return details. However, it does not explain the optional variables parameter, nor does it clarify whether the manifest must be pre-compiled or if the tool compiles it on the fly. These gaps leave the description incomplete for a tool with two parameters and a complex explanation output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It mentions the manifest ('for a manifest') but does not explain the required manifest_path parameter beyond the tool's focus, and it completely omits the optional 'variables' parameter. The description adds minimal value over the raw schema, leaving an important parameter undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Render `loom explain` for a manifest' – depicting the compiled plan as its own justification. It distinguishes itself from siblings like loom_compile by focusing on explanation and justification rather than compilation, and from iamf_validate/inspect by outputting a walkthrough of every target's routing and step rationale.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by describing the output ('walks every target', 'why it routed'), suggesting when a user would want to understand plan decisions. However, it does not explicitly state when to use this tool versus alternatives, nor any exclusions or prerequisites, so guidance remains implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iamf_inspectA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool parses OBUs and returns specific structural details, that it does not judge conformance, and notes MP4-specific behavior (edts/elst presence). This is transparent for an inspection tool, though it does not explicitly mention error behavior or non-mutating nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise: the first sentence states the core purpose, the second lists return details, and the third provides alternative guidance. Every sentence earns its place without fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one input parameter and an output schema exists, the description sufficiently explains what the tool does, what it returns, when to use it, and how it differs from siblings. It even includes format-specific details (MP4 edts/elst), making it complete for its simple scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single 'path' parameter with no description (0% coverage), but the description compensates by indicating the path points to an IAMF file ('an IAMF file's structure'). The implication is clear, though the description could have explicitly stated 'path to the IAMF file' for even greater precision.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Summarize an IAMF file's structure.' It explicitly differentiates from the sibling iamf_validate by saying 'without judging conformance' and 'this tool is for orientation,' making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs users to 'Use iamf_validate for conformance findings; this tool is for orientation,' providing a clear when-to-use versus alternative-guidelines. It also establishes the boundary by stating it does not judge conformance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iamf_validateA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden. It details the checks performed (structural L1, semantic L2, including S-407/S-408/S-409), the full report structure, parameter effects (strict, profile), and the L3 limitation, offering rich behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a clear opening, then a detailed but focused paragraph on checks and output, followed by parameter notes. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description covers input scope, validation levels, output report contents, parameter behavior, and an explicit limitation about L3. This is fully sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains the meaning of 'strict' and 'profile', and the first sentence makes it clear that 'path' is the target IAMF file. This compensates for the schema's lack of descriptions, though possible profile values are not enumerated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Validate an IAMF file (raw .iamf or IAMF-in-MP4) with iamf-sentinel', clearly stating the verb and resource. It distinguishes from sibling tools by focusing on conformance validation rather than inspection or compilation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it runs L1/L2 checks and explicitly notes 'L3 rendered-loudness QC ... is not exposed', providing a clear exclusion. However, it does not directly compare with sibling tools or give 'when to use' scenarios, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loom_compileA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: no execution, no file writes, ok=true/false return shapes, M-code diagnostics exactly as the CLI, and variable placeholder filling. This is comprehensive and adds significant context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary purpose. It then covers safety, return details, and parameter behavior in a logical structure. Every sentence earns its place, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description provides a complete picture: operation, side-effect-free nature, return format, diagnostics behavior, and parameter semantics. It is fully adequate for the tool's complexity and leaves no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explicitly explains the `variables` parameter as filling `{variable}` placeholders (CLI --var). `manifest_path` is self-explanatory given the tool's purpose, though it is not directly described. Partial compensation, with the most ambiguous parameter addressed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compiles an iamf-loom manifest (YAML/JSON) to a packaging plan. It uses a specific verb (compile), identifies the resource, and distinguishes from sibling tools by focusing on plan generation and validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it is validate-only, executes nothing, and writes no files. This implies usage for planning/side-effect-free inspection, but it does not explicitly compare with sibling tools like iamf_validate or loom_explain. A brief explicit alternative would make it a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.