Server Quality Checklist

Tool Scores

• create_image_configA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description must carry the burden. It explains the config structure, layer types, positioning, effects, max layers (50), and file path conventions. However, it does not mention idempotency, authentication, or rate limits, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is comprehensive but well-structured with clear sections (workflow, image structure, layer types, etc.) and front-loaded with purpose. It is slightly long but every section adds value; could be trimmed slightly but remains effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (nested objects, many options, no output schema), the description covers most aspects: layer types, effects, defaults, output formats, and workflow. It lacks some advanced details (e.g., error handling or limits enforcement) but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage, and the description adds significant detail beyond the schema, especially for complex parameters like 'layers' and 'defaults'. It explains layer types, positioning options, text features, and output formats, enriching the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it creates a JSON configuration for image generation. It distinguishes from sibling tools like create_video_config and references the workflow involving get_image_schema and validate_config, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides a workflow: get schema first, then create config, then validate if needed. It implicitly suggests when to use this tool versus siblings (e.g., get_image_schema for reading, validate_config for validation), though explicit exclusions are missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_configA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that the tool sends to API validation endpoint and checks various items, but does not mention side effects, rate limits, or potential cost implications beyond token estimation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with headings and bullet points, concise, and contains no unnecessary information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, and the description does not explain the return values or response format. It adequately describes the validation checks but lacks completeness regarding output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by specifying that the 'config' parameter comes from create_image_config or create_video_config, and clarifies that apiKey is optional if env var is set (though schema already notes optional).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool validates a job configuration against the jsoncut API, checking schema compliance, resource availability, estimated token cost, and errors. It distinguishes itself from sibling tools (create_image_config, etc.) by being a validation step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use: only when the user has provided actual media file paths, not placeholder paths. It also advises to always call after creating a configuration when real file paths are available.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_video_configA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It explains the tool creates a configuration object and describes the output in detail, including workflow and structure. It does not mention authorization, rate limits, or side effects, but for a non-destructive creation tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with clear sections (WORKFLOW, Video Structure, Layer Types, etc.). It is front-loaded with purpose and workflow. While every sentence contributes value, it could be slightly more concise, but the structure compensates.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (14 parameters, nested objects, many options), the description is comprehensive. It covers workflow, all layer types, audio options, positioning, transitions, and file path conventions. It also clarifies the return value and how to proceed with validation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds significant context beyond the schema, explaining video structure, layer types, audio options, positioning, and transitions. This greatly enhances understanding of how to use the parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a JSON configuration for video generation based on jsoncut documentation. It distinguishes from siblings like create_image_config and mentions the returned config can be used with validate_config or the API.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a WORKFLOW section that explicitly lists steps: first get schema, then create config, then validate config. It advises when to use validate_config (if user provided media file paths). However, it does not explicitly state when not to use this tool (e.g., for image configs), but sibling names imply the distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_image_schemaA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adequately conveys a safe read-only operation returning schema JSON. Notes the resource alternative. Could mention that no side effects or permissions are needed, but it's sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a brief introduction then notes. Slightly wordy with the IMPORTANT section, but still clear and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage order, alternative access, and what is returned (full JSON schema). Complete for a simple retrieval tool with no output schema needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so the description need not add param info. The baseline of 4 applies as zero-param tools require no further detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'complete JSON schema for image generation'. It distinguishes from sibling tools like get_video_schema (different resource) and create_image_config (different action).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to get this schema first when creating configurations, and mentions an alternative access method via resource. Provides clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_schemaA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it returns the full JSON Schema document, with no side effects or hidden behaviors; the description carries full burden given no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose, then adds key details and usage notes; each sentence is informative and well-structured without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description fully explains what the schema contains and why it's useful, meeting all contextual needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, baseline is 4; description adds context about the schema content without needing param details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it gets the complete JSON schema for video generation, distinguishing it from get_image_schema and other siblings that create or validate configs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using this tool first when creating video configurations and notes an alternative resource method, providing clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.