Server Quality Checklist

Tool Scores

• transcode_videoA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the job is queued and immediate return, but lacks details on cost, failure handling, output storage lifetime, or any destructive/read-only nature. This is insufficient for a tool that creates resources.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and contains no unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has nested parameters and creates async jobs with no output schema. The description mentions immediate return of a queued job but does not specify the response structure (e.g., job ID) or error handling. It also omits lifecycle details like temporary storage and expiration, leaving gaps for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is well-documented in the schema itself. The description adds no additional meaning beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create' and the resource 'video transcode job'. It specifies inputs (gs:// or https://) and output format, and distinguishes from siblings by mentioning related tools for tracking progress.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the tool returns immediately with a queued job and directs the user to use get_transcode, list_transcodes, or transcode_and_wait to follow progress. It implies asynchronous use versus synchronous with sibling transcode_and_wait. However, it does not explicitly state when not to use this tool or prerequisites like required permissions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_transcodesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses authentication requirement, pagination defaults, and optional filters, but lacks details on rate limits, ordering, or behavior when no results are returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences with no wasted words, effectively conveying the tool's purpose and key details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 5 optional parameters, the description adequately covers filters and pagination. Minor omission: ordering of results not mentioned, but overall complete for a list endpoint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds pagination defaults and overall filter purpose, but does not elaborate on parameter format or semantics beyond what schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (list), resource (transcode jobs), scope (authenticated account), and mentions optional filters and pagination, distinguishing it from siblings like get_transcode or transcode_video.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing jobs with filters, but does not explicitly state when to use this tool versus alternatives like get_transcode or cancel_transcode, nor does it mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcodeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden. It discloses the returned fields (status, output_url) but does not mention error behavior (e.g., non-existent ID), idempotency, or side effects. It lacks full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single 20-word sentence efficiently conveys the tool's purpose, action, and output. No unnecessary words, perfectly front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch operation with one parameter, the description covers the main output but omits response structure details (e.g., error format, full JSON example). Without output schema, more detail would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (id described as 'Transcode job UUID'). The description adds no meaning beyond the schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch'), the resource ('current state of a single transcode job'), and the specific data returned (status and output_url). This distinguishes it from siblings like list_transcodes (list all) or cancel_transcode (mutate).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context of sibling tools makes it clear that this tool is for fetching an individual job's state. However, it does not explicitly state when to use alternatives or provide exclusion cases, keeping it from a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcribeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that output_url appears when completed and implies read-only nature with 'fetch'. It does not mention error handling, authorization needs, or rate limits, which reduces transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with action and resource, no fluff. Every word contributes to understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch tool with one parameter and no output schema, the description covers the main output fields (status, output_url). It lacks details on other potential fields and does not guide usage relative to get_transcribe_download, but overall is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter described as 'Transcribe job UUID'. The description adds no further meaning beyond that, so it meets the baseline but does not compensate with extra detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Fetch' and resource 'state of a single transcribe job by ID', includes response details (status, output_url), and explicitly distinguishes from sibling tool get_transcode by noting it's for SRT generation jobs created via transcribe_audio.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides good context on when to use (to check state of a transcribe job) and acknowledges similarity to get_transcode for transcode jobs. However, it does not explicitly state when not to use or mention alternative tools like get_transcribe_download.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcribe_downloadA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses the URL's short-lived duration (10 minutes) and file type (SRT). However, it does not mention rate limits, idempotency, or error behavior if the job is not completed, which are relevant for a generation endpoint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences with no fluff. The first states the primary action and duration, the second sets precondition and a usage example. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, no output schema, no annotations), the description is complete enough. It covers purpose, precondition, and a practical integration tip. It lacks only minor behavioral details like error cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds context that the 'id' parameter corresponds to a completed transcribe job. The schema already describes the parameter as 'Completed transcribe job UUID', so the description provides marginal added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool 'Generate a short-lived signed HTTPS URL' for a completed transcribe job's SRT file, clearly distinguishing it from sibling tools like get_transcribe (which likely returns job metadata) and get_download_url (might be for other file types).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the precondition that the job must be in 'completed' status, and provides a practical use case in a transcode filter. It does not explicitly state when not to use the tool or list alternatives, but the context is sufficiently clear for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_transcodeA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full transparency burden. It discloses that cancellation only succeeds for queued/processing jobs and otherwise returns an error. However, it does not cover side effects, irreversibility, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. Essential information is front-loaded: what it does, when it works, and error cases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (one parameter, no output schema), the description provides sufficient context for an AI agent to understand usage, valid states, and error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema fully documents the single parameter. The description does not add additional meaning beyond the schema (e.g., format constraints). Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool cancels a transcode job, specifying the resource (transcode job) and action (cancel). Distinguishes from siblings like transcode_video (create) and get_transcode (read) by focusing on cancellation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states valid states for cancellation (queued or processing) and explains when not to use (completed, failed, cancelled jobs return error). Provides clear context, though no alternative tools are explicitly mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• confirm_uploadA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return values (fileUrl and probe metadata) but does not disclose validation behavior, error conditions, or side effects. With no annotations, a slightly higher bar would require mention of what happens if fileSize mismatch or other failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. Key information is front-loaded: 'Step 2 of the direct-upload flow.' Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains return values (fileUrl and probe metadata) and how to use them. Provides sufficient context for a two-step flow, linking to sibling tools. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers all parameters with descriptions. The main description adds value by clarifying that 'filename' is the storage object name (not original local filename) and that 'fileSize must match the size declared in request_upload_url'. This goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as 'Step 2 of the direct-upload flow' and states its specific verb+resource ('confirm upload'). It distinguishes from sibling tools like request_upload_url by positioning it as the follow-up step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies when to use ('after PUTting the file bytes to the URL returned by request_upload_url') and what to do with the result ('use the fileUrl as media_url for transcribe_audio or inputs[].url for transcode_video'). Lacks explicit when-not-to-use, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_upload_urlA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, description discloses key behaviors: short-lived presigned URL, PUT requirement with same Content-Type, expiry time. Could note error states but adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, logically structured steps. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description explains return type (presigned URL) and next step. Could specify exact return format but sufficient for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage; description adds context: filename as suffix, fileSize max 1.9GB, contentType must match PUT. Adds value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly identifies the tool as 'Step 1 of the direct-upload flow' returning a presigned URL, distinguishing it from sibling tools like 'confirm_upload'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description explains the workflow (this step, then PUT, then confirm_upload) and mentions the URL expiry (~15 min), giving clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcode_and_waitA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully explains the behavior: job creation, polling, timeout, and return of final job and download URL on completion. It could mention what happens on timeout or failure more explicitly, but overall it's transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently convey the purpose and usage. No redundant information, and key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core workflow and return value adequately. Given the complexity (6 params, nested objects, no output schema), it provides enough context for the agent to understand the tool's behavior and when to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add parameter details beyond the schema, which already contains detailed descriptions for each parameter including allowed values and constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it creates a transcode job, polls to completion, and returns the result with a download URL. It distinguishes itself from sibling tools like transcode_video and get_transcode by being a one-step convenience function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when you want the full transcode in one step without managing polling yourself,' providing clear guidance on when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_download_urlA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses the URL is short-lived (10 minutes), is an HTTPS URL, and requires job to be completed. This provides useful behavioral context beyond schema. Could mention if the URL is single-use or reusable, but still strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three precise sentences, front-loaded with purpose, condition, and rationale. No redundant information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (one param, no output schema, low complexity), the description covers purpose, condition, and alternative. Could be more explicit about return type, but overall adequate for an agent to decide to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, baseline is 3. Description adds meaningful context: explains why the id parameter is needed (completed transcode job) and justifies the tool's existence. This adds value beyond schema's basic type description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool generates a signed HTTPS URL for a completed transcode output file. The description uses specific verbs and resource ('generate a signed HTTPS URL for output file') and distinguishes it from sibling tools like get_transcode by focusing on download URL generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the tool should be used instead of the output_url field on the job object because it is a gs:// URL that HTTP clients cannot fetch directly. It also specifies the job must be in completed status, providing clear context for when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_audioA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses async nature (returns queued job envelope) and required polling. It doesn't cover error handling, rate limits, or file size constraints. Without annotations, this is good but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with primary action. No wasted words; every sentence provides essential workflow information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers the async process, polling workflow, download URL, and integration with transcode. Sufficient for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters. The description adds value by mentioning 'Whisper' and specifying preferred URL format (gs:// vs https://) beyond schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it generates SRT subtitles from audio/video URL using Whisper. It distinguishes from sibling tools by detailing the polling and download process, and mentions integration with transcode.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit instructions on when to use the tool: poll with `get_transcribe` for completion, fetch download URL with `get_transcribe_download`, and use SRT URL in transcode's subtitles filter. This clearly differentiates usage from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.