Server Quality Checklist

Tool Scores

• project_statusA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. 'Report' implies a read-only operation, but the description does not disclose side effects, required permissions, or return format. Minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every word contributes to the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description hints at the return value (list of stages) but does not specify format (e.g., array, string) or additional context like stage order. Adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the 'project' parameter lacks any description in the schema. The tool description does not elaborate on the parameter beyond its name, failing to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: reporting the stages (plan/lock/QC/manifest) for a project. The verb 'report' and specific stage names distinguish it from sibling action tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for checking stage progression but gives no explicit guidance on when to use this tool versus alternatives or when not to use it. Context from sibling names helps but is not provided in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lock_campaignA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose behavioral traits such as idempotency, side effects (e.g., overwriting existing locks), or error conditions. The term 'lock' implies a mutation, but no further context is given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a concise summary, followed by a structured list of arguments. It is not overly long, but slightly verbose with the 'Args:' block. Still, it earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The parameter descriptions are thorough, but behavioral context (e.g., what happens on repeated calls, return value) is missing. With no output schema and no annotations, the description leaves gaps in understanding the tool's full behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% coverage, so the description fully explains each parameter with examples (e.g., aspect ratio '2.39:1', hex palette example). This adds significant meaning beyond the schema's titles and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Lock the campaign look so every shot stays on-model.' This is a specific verb-resource combination that distinguishes it from sibling tools like 'assemble' or 'plan_shots'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for locking a look but provides no explicit guidance on when to use it versus alternatives or when not to use it. No exclusions or alternative tools are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan_shotsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that the tool saves to plan.json and returns the plan. However, it does not mention idempotency, overwrite behavior, or required permissions. The description adds some context but lacks comprehensive behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with a clear summary sentence followed by details. The Args section is structured but could be more readable with bullet points. Overall, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters and no output schema, the description adequately covers purpose, parameters, and side effects. It explains what the tool does and returns. Lacks error handling details but sufficient for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description includes an Args section that explains each parameter's meaning: brief (plain-language description), project (scoping), n_shots (target number). This adds significant value beyond the schema's minimal titles and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: turning a brief into a structured block-method shot plan. It specifies actions like planning shots and saving to plan.json. This distinguishes it from sibling tools (assemble, lock_campaign, etc.) which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by saying 'before any pixel is generated', but does not explicitly state when to use this tool versus alternatives or provide any exclusions. There is no direct guidance on when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qc_stillA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It mentions saving the verdict but does not specify where or if any state is modified, what authorization is needed, or whether the operation is read-only. This lack of detail leaves potential side effects ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loading the purpose and then providing necessary details in a structured bullet-style Args section. No extraneous sentences, though the formatting could be slightly more compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, no output schema, and no annotations, the description covers the key aspects: what it does, how scoring works, pass condition, and post-QC action (re-roll with fix_suggestion). It lacks detail on the return structure but is adequate for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite the input schema having 0% parameter descriptions, the description includes an Args section that explains each parameter (project, image, shot_id, threshold) with sufficient meaning. The default value for threshold is noted. This compensates for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs style-drift QC on a generated still, scoring intent/look/character on a 0-100 scale and returning a verdict. This verb-resource combination distinguishes it from sibling tools (assemble, lock_campaign, plan_shots, project_status), which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that an agent re-rolls the still when pass is false, using fix_suggestion, implying when to use this tool. Although it doesn't explicitly exclude usage scenarios or compare with alternatives, the context makes the appropriate use case clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assembleA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses that the tool is a blueprint and the NLE cut is manual. It mentions optional clip attachment but does not describe side effects like overwriting or append behavior, permissions, or rate limits. The absence of destructive/read-only hints means the agent must infer behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with no wasted sentences. It front-loads the core purpose and includes a standard Args block. Minor improvement could be separating the background info from parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter tool with no output schema, the description covers purpose and parameters adequately. However, it does not describe the return value (the manifest structure), error conditions, or explicitly state that a plan must exist beforehand.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% coverage (no parameter descriptions), but the description adds meaning: 'project: project name' and 'clips: optional rendered-clip refs keyed by shot_id.' This clarifies the purpose of the clips parameter far beyond the schema's anyOf definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds a cut manifest with shot order, durations, and audio notes. This specific verb+resource combination distinguishes it from sibling tools like plan_shots or lock_campaign.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the tool uses a plan for order/timing and that clips are optional. It also clarifies that the actual NLE cut remains manual in v1, providing context on when this is an intermediate step. However, no explicit when-not-to-use or alternative tools are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.