Server Quality Checklist

Tool Scores

• create_video_with_imageB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the image guides visual style, but does not disclose asynchronous behavior, required permissions, rate limits, or any side effects beyond creating a video. The tool mutates state, but this is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action, no filler. Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex media generation tool with 6 parameters and no output schema, this description is insufficient. It does not explain the relationship between prompt and image, expected output format, or mention that video creation is asynchronous (evidenced by sibling 'wait_for_video'). The description leaves the user guessing about the full workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter having a description. The tool description adds value by explaining that the image guides visual style, which supplements the image_url/base64 parameter semantics, but this is minimal additional meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a video from an image as the first frame, using a specific verb ('Create') and resource ('video using an image'). This distinguishes it from sibling tool 'create_video' which likely creates video from text only.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an image is available to guide the video, but does not explicitly compare to alternatives like 'create_video' or state conditions when this tool should be preferred. The context is inferable but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_videoB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry behavioral context. It adds a useful disclosure that authentication is handled internally and the API key is not exposed, but it omits other behavioral details such as overwrite behavior or what happens if the video is not yet complete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the purpose and add a relevant security detail. No wasted words, though 'Download and save' is slightly redundant but acceptable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should at least hint at return behavior, but it does not. It also fails to clarify how this tool differs from 'download_video', leaving a significant selection gap despite adequate schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all three parameters, including the variant enum and output_path default. The description adds no parameter-level detail, so the schema carries the burden; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Download') and identifies the object ('a completed video' saved 'to a local file'), making the core action clear. However, it does not distinguish itself from the sibling tool 'download_video', so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'completed video' implies it should be used only after the video has finished processing, and 'handles authentication internally' suggests a reason to prefer it. But there is no explicit when-not or alternative comparison, especially with the sibling download_video.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_video_statusA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses return values and the list of statuses, which is helpful. However, it does not mention potential error conditions (e.g., not found, invalid ID), rate limits, idempotency, or confirm that it is a read-only operation beyond the verb 'Get'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just two sentences, concise and front-loaded with the core action. Every word adds value, and it avoids repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only status-checking tool with one parameter and no output schema, the description covers the essential information: what it does, what it returns, and the possible statuses. It is slightly lacking in behavioral edge cases (e.g., error handling) but is otherwise complete for basic selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a complete description for video_id ('The ID of the video job to check') at 100% coverage. The tool description adds no additional parameter semantics, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb 'Get' and resource 'current status of a video generation job'. It also enumerates the return values (progress percentage and status), making it distinct from sibling tools like wait_for_video, which implies blocking or waiting behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as wait_for_video. No mention of polling patterns, whether to call this repeatedly, or that this is non-blocking. The description only explains the tool's purpose without any contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wait_for_videoA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral transparency. It clearly states the polling behavior and that it returns the final status. However, it does not explain what happens when timeout_seconds is exceeded (e.g., whether it throws, returns a timeout status, or silently fails) or how invalid video_id values are handled. These are important edge cases for an agent to predict.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary verb 'Poll', and every sentence adds value: the first defines the action, the second articulates the use case. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple polling tool, the description covers the core purpose, usage context, and basic return value. It lacks details about timeout behavior and possible final statuses, but these are partially covered by the schema. It's reasonably complete given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters (video_id, timeout_seconds, poll_interval_seconds) adequately. The description itself adds no parameter-specific meaning, so it meets the baseline but does not go beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Poll a video job until it completes or fails') with a clear resource and outcome. It stops short of explicitly distinguishing itself from siblings like get_video_status, which likely performs a single status check, but the 'until it completes or fails' and 'without manual polling' infer that distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly frames when to use the tool: 'Useful for waiting on video generation without manual polling.' This gives clear context for using it as a blocking wait rather than a one-off status check. However, it does not mention alternatives or when not to use it (e.g., when only a single status check is needed).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_videosA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided, so the description carries the burden of conveying safety. The verb 'List' implies a read-only operation with no destructive side effects, and 'pagination support' hints at paged responses. However, it doesn't explicitly state the return format, default ordering, or any account-specific caveats, leaving some behavioral aspects implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that wastes no words. It conveys the essential information immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward list operation, the description covers the core purpose, scope, and pagination feature. The schema documents all parameters. However, with no output schema, the description does not describe the return shape, but this is not essential given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with descriptions for all three parameters (after, limit, order), including defaults and constraints. The description reinforces the concept of pagination but adds no additional parameter-specific meaning beyond what the schema offers, warranting the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List'), the resource ('video generation jobs'), the scope ('for your account'), and a key feature ('pagination support'). This distinguishes it from sibling tools like get_video_status or delete_video.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates this tool is for listing all video generation jobs, which is the natural use case. However, it does not explicitly mention when to prefer this over alternatives, such as get_video_status for individual jobs, so it stops short of full alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remix_videoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does disclose that the tool preserves the original structure while applying changes, which is useful. However, it does not mention potential side effects (whether the original is modified or a new video is created), asynchronous behavior, or required permissions. Some key behavioral aspects are left assumed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence states the core function, and the second adds a practical usage tip. Highly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with only 2 parameters and no output schema, the description is adequately complete. It covers what the tool does and provides guidance on effective use. It could benefit from indicating whether the operation is asynchronous or returns a new video ID, but given the sibling tools (e.g., wait_for_video), this is implied. Overall, sufficient for an agent to understand and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The tool description itself does not add parameter-specific meaning beyond what the schema provides. The schema already includes guidance like 'Make single, focused changes for best results,' which the description does not repeat or enhance. Therefore, baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it creates a variation of an existing completed video with targeted adjustments, using a specific verb ('create a variation') and resource ('existing completed video'). Distinguishes itself from sibling tools like create_video and create_video_with_image by focusing on modifying existing videos rather than creating new ones.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on when to use: for existing completed videos and best for single, well-defined modifications. Implies not for new creations or complex changes but does not explicitly name alternatives or exclusions. However, the context given is sufficient for an agent to make a reasonable choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_videoA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds the critical note 'This action cannot be undone,' which tells the agent the operation is irreversible. This is valuable context beyond the schema, though it doesn't mention other potential side effects or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences: the first states the action, and the second adds a crucial caveat. Every word earns its place, with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema), the description provides complete context: what it does, what it affects (storage), and that it's irreversible. No additional information is needed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents the only parameter (video_id) with a description, achieving 100% schema coverage. The tool description adds no additional parameter semantics, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Delete a video from OpenAI's storage' with a specific verb and resource, making the purpose unambiguous. It clearly distinguishes from sibling tools like create_video, list_videos, and download_video, none of which perform deletion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied: use this tool when you want to permanently remove a video. However, there is no explicit guidance about when not to use it or comparison with alternatives. Since there is no other delete tool among siblings, the context is clear but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_videoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the transparency burden. It discloses key behaviors: returns a job ID, is asynchronous, and may take several minutes. This is critical for setting user expectations. It does not elaborate on failure modes or rate limits, but the disclosed traits go beyond basic schema and give meaningful operational insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, immediately leading with the core purpose. Each sentence adds necessary value: what the tool does, what it returns, and how it behaves. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential workflow: creating a job and receiving a job ID for later status checks and downloads. It accounts for the asynchronous nature and the lack of an output schema. It doesn't mention all parameter choices, but that's already in the schema. Minor gap: it does not explicitly reference sibling tools like get_video_status or download_video, though the job ID mention implies their use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for all four parameters, including enums and defaults. The description adds no extra parameter-specific information beyond what the schema already provides, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a new video generation job using OpenAI's Sora model.' It specifies the verb ('create'), resource ('video generation job'), and model context. It also differentiates from sibling tools by focusing on prompt-based creation, while a sibling like create_video_with_image likely handles image-based input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this is the tool for creating a new video generation job from a text prompt using Sora. It does not mention explicit alternatives or exclusions (e.g., using create_video_with_image for image-based prompts), but the context is unambiguously about starting fresh from text. This aligns with a clear usage scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_videoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It goes beyond a simple 'get download URL' by revealing that the URL is time-limited (valid for 1 hour) and only works for videos with 'completed' status. These are non-obvious behavioral traits that help the agent handle the tool correctly. It does not mention error handling or authentication, but the key behavioral constraints are effectively conveyed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two sentences that immediately state the primary function, then add the two most crucial constraints (URL expiry and status requirement). Every sentence earns its place, with no redundant or filler content. It is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's modest complexity (2 parameters, no output schema, no annotations), the description covers the essential contextual aspects: what it returns (a download URL), when it is valid (completed status), and a key time-sensitive behavior (1-hour validity). It does not cover error cases or advanced usage, but for a simple download-URL tool, the provided context is sufficient for an agent to select and invoke it correctly. A minor gap is the lack of guidance on what happens upon URL expiry, but overall it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% description coverage for both parameters, including detailed meaning for 'variant' (video, thumbnail, spritesheet). The tool description does not add further parameter-level detail beyond what the schema already states. According to the baseline, when schema coverage is high, a score of 3 is appropriate because the description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Get a download URL for a completed video'), clearly stating what the tool does. It distinguishes this from siblings by adding a scoped condition ('completed' status) and a key detail (URL valid for 1 hour). This makes the purpose unambiguous and distinct from related tools like get_video_status or wait_for_video.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: it only works for videos with 'completed' status, which implicitly tells the agent to first check status (via get_video_status or wait_for_video) before calling download_video. It does not explicitly name alternatives or exclusions, but the status prerequisite is strong contextual guidance. Given the sibling list includes status-checking tools, this is helpful but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.