Server Quality Checklist

Tool Scores

• get_jobB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'Read' implies a non-mutating operation, but the description does not disclose error behavior for missing/unknown job IDs, whether it polls or returns immediately, or what 'state' includes. Beyond its purpose, it adds almost no behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with no filler or redundancy. It is appropriately front-loaded, though it is terse enough that it sacrifices some useful detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter getter with an output schema, the description is minimally adequate. It clearly states the core action, and the parameter is self-explanatory. However, it lacks guidance on job_id provenance and error cases, which would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required parameter, job_id, with 0% description coverage. The description does not mention job_id, where it comes from, or how it relates to other tools, so it adds no semantic value beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb, 'Read,' with a specific resource, 'the current state of a generation job.' This distinguishes it from siblings: list_jobs lists jobs, cancel_job mutates, generate_media creates, and get_job reads a single job.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the singular 'a generation job' and the sibling names, but there is no explicit guidance on when to choose this over list_jobs or how to obtain a job_id. The description does not state exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jobsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does state ordering ('newest first') and optional state filtering, which adds useful behavior beyond the schema. However, it does not clarify what 'recent' means, whether results are paginated, or any other runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every phrase adds meaning, and it is appropriately sized for a simple list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema reduces the need to describe return values. Yet the lack of parameter detail, especially valid state values and limit semantics, means an agent may still guess when calling. It is adequate for a basic list tool but has clear informational gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It mentions the state filter but does not explain valid state values or the meaning/limits of the limit parameter. This is partial coverage that leaves important parameter semantics undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the verb ('List'), the resource ('jobs'), and the core behavior (newest first, optional state filtering). This distinguishes it from siblings like get_job and cancel_job without confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use list_jobs versus alternatives such as get_job or cancel_job. There is no mention of when not to use it or which sibling handles related but different lookups.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_jobA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It usefully discloses that cancellation is 'best effort provider-side' and that the tool is 'always terminal here', giving the agent a clear mental model of the operation's guarantees and limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The primary action is stated first, and the additional behavioral nuance is packed efficiently into the second sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single required parameter and an output schema, the description covers the core action and key behavior, but it omits important operational details like what happens if the job is already terminal or the job_id is invalid. It is minimally viable but leaves the parameter semantics and error behavior unaddressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, job_id, has no schema description (0% coverage), and the tool description does not explain its format, origin, or how to obtain a valid value. The description only mentions 'a running job', leaving the agent to infer what job_id should be.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Cancel') and a clear resource ('a running job'), immediately distinguishing it from siblings like get_job and list_jobs. The behavior is not confused with any other tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives, such as checking status with get_job first or handling jobs that are already terminal. The intended context is only implied by the phrase 'running job'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_mediaA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and does it well: it discloses that the call returns before the job is done, enumerates terminal states, and explains default routing behavior. This goes well beyond the schema, though it does not mention cancellation or lifecycle side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, all high-signal. The most important action is front-loaded, the async behavior comes immediately after, and the backend guidance is a compact decision rule. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete on the job lifecycle and backend routing, and an output schema exists for the job-handle return. It is less complete for a 7-parameter tool because it never explains modality-specific parameter applicability (e.g., duration_s vs width/height) or require_tags, and does not point to list_capabilities for discovery.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameters, but it only adds semantics for backend. prompt, modality, width, height, duration_s, and require_tags are left to their types/names, and require_tags in particular has no hint of its meaning or effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a concrete verb and resource: 'Start a generation and return a job handle.' This clearly separates it from siblings like get_job, cancel_job, and list_jobs, all of which are later referenced or implied by the async lifecycle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the async workflow explicit: call this to start the job, then poll get_job with the returned job_id. It also gives concrete guidance on when to set backend vs leave it unset. It does not, however, spell out when to prefer list_capabilities first or when not to call this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_capabilitiesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It adequately discloses the read-only nature of 'list', the complete scope ('every model'), and the returned attributes (cost and latency). It does not mention possible response size or data freshness, but these are minor for a simple no-parameter listing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core verb and resource while adding the key return data (cost and latency). Every word is informative and there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter read-only listing with an output schema, the description covers the main facts needed to invoke the tool correctly. A short note on when to use it relative to sibling tools would improve completeness, but nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters, and the schema already reflects this with an empty object. There is nothing for the description to add about parameters, so the baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a concrete verb, 'List', and identifies the exact resource: 'every model this server can route to'. It also states what information is returned (cost and latency). This clearly distinguishes it from siblings like list_jobs, which target a different resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus alternatives. The resource is clear, but the description does not state when it should be invoked, such as before generating media, or when a different listing tool like list_jobs would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.