Server Quality Checklist

Tool Scores

• delete_videoA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the key behavioral trait that deletion is permanent, which is crucial for a destructive operation, but lacks additional context such as error behavior or permissions. Since no annotations are provided, the description carries the burden, but 'permanent' provides some transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff, front-loading the action and adding a critical warning about permanence. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete tool with one parameter and no output schema, the description is adequately complete, covering the action, scope, and permanence. It could add details about error handling or idempotency, but these are not necessary for such a straightforward operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully documents video_id as 'The video job ID to delete', and the description does not add additional parameter semantics. With 100% schema coverage, the description adds no extra meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deletes a video from OpenAI's storage with the verb 'Delete' and resource 'video', and it distinguishes from sibling tools (create, get, download, list, remix) by being the only delete operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives or any prerequisites. It only states the action without context for when deletion is appropriate or any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses possible status values and mentions progress, which is useful. However, it does not explicitly state that this is a read-only operation, how errors are handled, or what the response structure will look like. Moderate transparency for a simple status-check tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the action and resource. The status value enumeration is informative and adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter status tool, the description covers the core operation and enumerates status values. However, without an output schema or annotations, it does not clarify what 'progress' means or describe the response format, leaving some ambiguity. It is still largely complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the schema description already states that video_id is returned from create_video. The tool description adds no additional parameter semantics beyond what the schema provides, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('retrieve') and clearly identifies the resource ('current status and progress of a video generation job'). It distinguishes itself from sibling tools by focusing on status polling, and the enumeration of status values reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after creating a video job but does not explicitly state when to use this tool or exclude alternatives. No mention of 'use after create_video' or comparison with sibling tools, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remix_videoA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds a meaningful behavioral trait ('Preserves structure and composition'), but does not mention permissions, async behavior, return values, or side effects on the original video. Still, it offers more than a tautological statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and scope. The second sentence adds a valuable behavioral note without redundancy. No filler or repetition of schema fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a mutation with no output schema and no annotations. The description explains the main behavior but omits details about what the caller receives (e.g., video ID, status), whether processing is synchronous, and any specific requirements beyond a completed video. It is minimally sufficient but leaves key operational gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds some context for the 'prompt' parameter ('targeted adjustments') but adds no new meaning for 'video_id' beyond the schema. It does not compensate further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: creating a new video by remixing an existing completed video with targeted adjustments. It explicitly mentions the base resource ('existing completed video') and the action ('remixing'), distinguishing it from siblings like create_video, which would start from scratch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use when you have an existing completed video and want to apply targeted modifications while preserving structure. It implicitly distinguishes from create_video by requiring a completed video, though it does not explicitly name alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_videosA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses pagination support, the 'all' scope, and that it returns metadata. It does not describe the metadata fields, default sort order, or any edge cases (e.g., empty pages), leaving some behavior implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states the action and scope, the second states the purpose. Every word earns its place; no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no output schema, the description adequately covers purpose and pagination. It could mention the default ordering or the exact metadata fields returned, but the schema covers parameter details, making this sufficient for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides thorough descriptions for all three parameters (after, limit, order) with default values and max constraints, so the description does not need to repeat them. The description's mention of pagination aligns with the 'after' cursor parameter, adding minimal extra context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('video jobs'), and 'with pagination support' further clarifies scope. This clearly distinguishes it from siblings like create_video, delete_video, and get_video_status, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit usage context: 'for enumeration, dashboards, or housekeeping.' However, it does not explicitly contrast with alternatives like get_video_status, which targets a single video's status, so the guidance is clear but lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_videoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the async nature (returns a job object, monitor via status/webhooks) and hints at the workflow. This goes beyond basic, though it omits potential details like rate limits, which are not essential for a simple create operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all informative and non-redundant. Front-loaded with the primary purpose, followed by return value and monitoring guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a create tool with 5 params and no output schema, the description sufficiently covers the purpose, async workflow, and how to monitor completion. It could elaborate on response fields, but get_video_status is mentioned for that. Overall, it is well-rounded for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed descriptions for all 5 parameters. The description adds no parameter-specific guidance beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Start a new video generation job with Sora', which is a specific verb+resource. It distinguishes from siblings (delete_video, get_video_status, download_video, list_videos, remix_video) by focusing on creation, and adds the return of a job object.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context as the entry point for video creation and explicitly directs the user to poll get_video_status or use webhooks for monitoring. However, it does not explicitly state when not to use this tool versus alternatives, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_videoA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose the return type (base64-encoded binary data) and the precondition, which is useful. However, it does not specify behavior on failure, such as what error occurs if status is not completed, or whether the download is read-only. These are minor gaps for a download operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first sentence states the core action and variants; the second provides the critical condition and return format. There is no filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple download tool with only two parameters, this description is complete. It covers the resource types, the precondition, and the return format. The absence of an output schema is compensated by the explicit mention of base64-encoded data. Sibling tools are clearly different in purpose, so no additional context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description goes beyond by mapping each variant to its format (video=MP4, thumbnail=WebP, spritesheet=JPEG) and adds the completed-status condition for video_id. This enriches the schema's terse definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Download' and clearly identifies the resources (video, thumbnail, spritesheet) with their formats (MP4, WebP, JPEG). This completely distinguishes it from sibling tools like delete_video or create_video.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the explicit precondition 'Only works when status is completed', which guides the agent on when it is appropriate to call. It does not explicitly name alternatives like get_video_status for checking status, but the condition implies the need to check first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.