Server Quality Checklist

Tool Scores

• check_video_statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the status values, the success condition for video_url, and expected latency. It does not, however, describe behavior on failure (e.g., possible error fields), invalid job IDs, or whether the operation is read-only (implicit but not stated). This covers the basics but leaves some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. The purpose is stated first, followed by status semantics and practical polling advice. Every sentence earns its place; nothing is over-explained.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter polling tool with no output schema and no annotations, the description conveys the key response elements (status, video_url on success) and timing. It lacks details on failure responses, error handling, or what happens with an invalid job ID, making it adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for job_id already states 'The job id returned by generate_video', and coverage is 100%. The description adds no extra parameter details beyond this; it only reinforces the generate_video link. Baseline 3 is appropriate since the schema carries the semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Poll' with the resource 'status of a video generation job', and immediately ties it to generate_video, making it distinct from siblings like list_video_jobs (lists all) and generate_video (creates). The action and scope are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states the tool is for jobs started with generate_video and provides concrete polling guidance ('Poll every few seconds') with expected time ranges. However, it does not explicitly name alternatives or state when not to use it, though the context strongly implies it's for a single job's status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_videoA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full disclosure burden and largely succeeds: it reveals the async return contract (job id + 'queued' status), the polling requirement, and — critically — the real financial consequence ('Deducts real credits from the caller's KaiRouter account balance when the job is accepted'). It misses minor items like rate limits, retry safety, or typical job duration, but the expensive-traits an agent needs to fear are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, roughly 85 words, each earning its place: what it does, what it returns, how to check results, and the cost warning. The structure is perfectly front-loaded — purpose first, operational warning last — with zero filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity tool (async job, external service, real credits, no output schema, no annotations), the description covers the critical operational contract: launcher semantics, polling pattern, statuses, and billing side effects. Remaining gaps are real but minor — no idempotency/retry guidance (relevant when retries could double-charge) and no time-to-completion estimate to relay to users. Still well above the minimum viable bar.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline of 3 applies — and the schema genuinely earns it with rich per-field detail (defaults, examples, the data: URI hint on image_url, and the cost-estimation context on input_duration_secs). The tool description itself adds only marginal model-version context (e.g., 'BytePlus Seedance 2.0') that isn't already encoded in the schema. Correct baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb-resource pairing ('Start an async video generation job on KaiRouter') and immediately demonstrates differentiation from siblings by naming them: 'see list_video_models' and 'Call check_video_status'. The async contract ('Returns immediately... does NOT wait') is stated up front. An agent could not confuse this with its siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit, condition-driven routing: use list_video_models when you need valid ids, use check_video_status with the returned id to poll. The polling workflow (poll until 'succeeded' or 'failed') is spelled out, leaving no ambiguity about when to invoke this tool vs. its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_video_jobsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses key behaviors: caller scoping, a 50-job limit, and recency ordering. While it does not explicitly state read-only, 'list' inherently implies no mutation, and the given details add useful context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. The first sentence front-loads the core action and scope; the second justifies use cases. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless list tool, the description covers the essential aspects: what is listed, the limit, ordering, and practical use cases. It does not describe the response fields, but without an output schema, that is a minor gap and does not hinder correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description correctly focuses on what the tool does rather than parameters, as none exist. No additional parameter explanation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the specific verb 'List', a clear resource ('video generation jobs'), and a unique scope (caller's recent jobs, up to 50). It distinguishes itself from sibling tools like list_video_models and check_video_status by focusing on jobs, not models or status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly gives use cases: 'recovering a job id that was lost, or reviewing recent generations.' It does not name alternatives or exclusions, but the context is clear enough for an agent to choose this tool when needing a job list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_video_modelsA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that model availability changes over time, which is a useful behavioral trait. However, it doesn't mention whether the tool is read-only, the response format, or any rate limits. Since the tool is clearly a listing operation, the read-only nature is implied but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and includes a practical usage hint. Every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool, the description is complete: it states what is returned (id, pricing, health), why to call it (find valid model id), and the dynamic nature of the data. It doesn't describe the output schema, but no output schema is provided, so the description covers the essentials.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description doesn't need to explain parameter semantics. The baseline for 0 params is 4, and the description appropriately focuses on what the tool returns rather than parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists video generation models on KaiRouter, specifies the model family (BytePlus Seedance), and mentions the included details (id, pricing, provider health). It distinguishes itself from siblings by explicitly positioning it as the first step before generate_video.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to call this tool first to find a valid model id for generate_video, and warns not to assume a model id without checking. This provides clear when-to-use guidance and implicitly differentiates from the sibling tools (list_video_jobs, check_video_status) which serve different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.