Server Quality Checklist

Tool Scores

• generate_videoC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of disclosing behavioral traits. It does not mention that video generation is asynchronous, that it returns a project ID, that there may be costs or API limits, or that the process may take significant time. The only external reference is a URL for the full schema, but critical behavioral context is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus a link, which is concise. However, it lacks structure: no bullet points, no clear separation of key points. It is not front-loaded with the most critical information (e.g., the required 'scenes' parameter). It is adequate but not well-optimized for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the tool (13 parameters, deeply nested objects, required scenes) and the absence of an output schema, the description is insufficiently complete. It does not explain the return value, the asynchronous nature, the cost implications, or how to handle the response. The tool relies entirely on external documentation, which is poor practice for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and each parameter has a description, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides. It lists element types but that is also covered in the schema. Thus, it does not improve parameter understanding beyond the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a video project using the json2video API and lists the types of elements that can be included (text, images, video, etc.). It effectively communicates the primary purpose. However, it does not differentiate from sibling tools like create_template, which also involves creating something, though the distinction is somewhat implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as create_template or get_video_status. It does not mention prerequisites, context, or exclusions. The agent receives no help in deciding whether to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It fails to disclose behavioral traits like idempotency, error handling (e.g., duplicate name), authentication requirements beyond the optional apiKey parameter, or whether the created template is returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely brief (one sentence), but it lacks structure and does not provide any useful elaboration. It is under-specified rather than concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and annotations, the description should explain what the tool returns (e.g., created template ID) or side effects. It does not, leaving the agent uninformed about the outcome of the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all parameters have descriptions). The description adds no additional information beyond the schema, so it meets the baseline but does not enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and the resource ('template') with scope ('in json2video'), differentiating it from siblings like get_template (retrieve) and list_templates (list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, such as when to create a new template versus using an existing one. No preconditions or dependencies are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_templateB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden, but it only indicates a read operation without disclosing permissions, side effects, or error conditions. Adequate for a simple retrieval, but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no redundancy, efficiently conveys the core purpose without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with only two parameters and no output schema, but the description does not specify what details are returned, leaving the agent without complete context on the return value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the parameter names and their schema descriptions, which are already clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get template details from json2video' clearly states the action (get) and resource (template details), and implicitly distinguishes from sibling tools like list_templates, but no explicit differentiation is provided.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use versus alternatives; the context implies use when you have a template name and need details, but no when-not or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_statusB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Lacks details on what 'status or result' entails (e.g., polling behavior, response format, mutability). Insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded, and no fluff. Could be slightly more detailed without harming conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Minimal coverage for a simple tool. Lacks explanation of output format or status values. Adequate for a straightforward status check but incomplete for an agent to interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. Description adds no additional semantics beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Get' and resource 'status or result of a generated video'. Distinct from siblings which are about template creation/list and video generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage as a follow-up to generate_video, but no explicit guidance on when to use vs. alternatives or exclusions. No mention of siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_templatesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states a read operation but does not disclose behavior like output format, pagination, authentication requirements (optional apiKey), or any limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence, front-loaded with key information. It could be slightly more informative without sacrificing conciseness, but currently it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (1 optional param, no output schema), the description is adequate but minimal. It does not hint at the return structure, which would be useful since no output schema is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing full parameter meaning. The description adds no further semantic value beyond what the schema already includes for the optional apiKey parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (List) and resource (all available templates from json2video). It distinguishes from sibling tools like get_template (single) and create_template (write operation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. However, siblings imply this is for getting an overview vs. specific template retrieval. Lacks exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.