Server Quality Checklist

Tool Scores

• dispatchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With zero annotations, the description carries full burden but only mentions that it returns {job_id}. It does not disclose whether the operation is synchronous or asynchronous, side effects, authentication needs, rate limits, or error behavior. This is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the action and result. It avoids fluff, though it could be slightly expanded without losing conciseness. Overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, including a timeout and context files), the description is incomplete. It doesn't explain how timeout or context_files affect behavior, or what happens on failure. The presence of poll/wait siblings implies async, but this is not stated. More detail is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain meaning. It only clarifies that 'agent' is one of three worker types. The other parameters (cwd, prompt, timeout_ms, context_files) remain unexplained, providing little value beyond the schema structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (dispatch a subtask), the target (code agent worker with explicit options opencode, codex, claude), and the return value (job_id). It distinguishes well from siblings like poll, list_jobs, cancel, wait which are about monitoring or managing jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use dispatch versus alternatives like when to combine with poll/wait, or prerequisites (e.g., agent availability). It only states what it does, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jobsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It states the ordering ('newest first') but does not mention pagination, rate limits, scoping (user vs all jobs), or authentication requirements. It's adequate but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with purpose and key details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (which explains return values) and no annotations, the description covers the basics but lacks completeness about scope (which jobs?) and presence of pagination. For a simple list tool it's acceptable but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for the 'state' parameter by listing possible values (pending|running|done|error|cancelled). However, it does not explain the 'limit' parameter beyond what the schema already provides (default 20).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List jobs'), the ordering ('newest first'), and an optional filter. This differentiates it from sibling tools (poll, dispatch, cancel, wait) which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions an optional state filter with enumerated values, giving context for filtering. However, it does not explicitly state when to use this tool versus siblings, though the action is distinct enough that it's implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pollA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It discloses the return structure (state, result, etc.), but does not discuss error handling, idempotency, or behavior for non-existent jobs. Still, it is straightforward for a simple polling tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with verb and resource, lists return fields. No fluff, every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (1 param, no nested objects) and presence of an output schema (return fields provided), the description is fairly complete. It covers the essential information for a polling tool, though it omits possible states or error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage of parameters (1 param, job_id), but the description does not add any meaning to it beyond the schema's title 'Job Id'. With 0% schema_description_coverage, the description fails to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Poll' and the resource 'job state', and distinguishes from siblings like list_jobs (list all) and wait (blocking). The return fields are explicitly listed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use after dispatching a job, but does not explicitly state when to use poll versus wait or other siblings. No when-not-to-use or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancelA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully discloses behavior: sends SIGTERM then SIGKILL after 5s, and returns {ok, prev_state}. This is comprehensive for a cancellation action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. Critical information (signal sequence, return format) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simplicity (1 param), description covers purpose, behavior, and return value. Minor gap: no mention of error cases (e.g., job not found), but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. Description does not explain job_id beyond its name, though job_id is self-explanatory. Baseline for low coverage requires compensation, which is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool cancels a job using specific verb and resource. It distinguishes from sibling tools like poll, dispatch, list_jobs, and wait.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied (use when you want to cancel a job) but no explicit guidance on when not to use or alternatives. Lacks prerequisites or context for decision.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• waitA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses server-side blocking, return shape same as poll, timeout behavior, and reusability. It does not mention rate limits or authentication, but for a wait function, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose, and every sentence adds value. Zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple wait tool with an output schema (implied from 'same shape as poll'), the description covers the essential behavior, contrasts with a key sibling, and explains edge cases. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implicitly references job_id and timeout_ms but adds no extra detail beyond their names (e.g., format constraints, allowed values). The parameters are self-explanatory, but the description does not enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool blocks until a terminal state or timeout, and it distinguishes itself from the sibling 'poll' by replacing multiple poll calls. The verb 'wait' and the resource 'job' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts with poll (reduces polling cost) and explains the timeout behavior (returns current state, can wait again). It clearly states when to use this tool, though it could more explicitly mention when not to use it (e.g., if you need periodic status updates).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.