Server Quality Checklist

Tool Scores

• codex_cancelA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals the mechanism (killing tmux session), indicating a forceful termination. However, with no annotations, it fails to disclose side effects like data loss or reversibility, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and front-loaded with the action. However, it is overly terse and could benefit from additional context without being wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple cancellation tool with one parameter and an output schema, the description explains the purpose and mechanism. However, the lack of parameter documentation reduces completeness, though the output schema may compensate for return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage for the single required parameter 'task_id'. The description does not explain what task_id represents or how to obtain it, forcing the agent to infer from the tool name alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool cancels a running Codex task by killing its tmux session. This specific verb and resource distinguish it from siblings like codex_dispatch (submit tasks) and codex_status (check status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for canceling running tasks, but provides no explicit guidance on when to use it versus alternatives like waiting for completion or other cancellation methods. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_dispatchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses key behaviors: non-blocking, persistent tmux session, survival of disconnects. However, it omits details on error handling, concurrency, and sandbox side effects (e.g., full-access creates git worktree).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core action and returning immediately. Every word adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with low schema coverage and no annotations, the description is minimal. It covers the main purpose and lifecycle, but lacks details on input parameters and output (though output schema exists). It is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 20% schema coverage (only sandbox has a description), the description adds no information about any parameters. The 5 parameters, including required ones like prompt, cwd, and topic, are left unexplained, offering no additional meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool dispatches a long-running Codex task to background, returns immediately, and runs in a persistent tmux session. It distinguishes from siblings by referencing codex_status and codex_cancel.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises to use codex_status for progress and codex_cancel for stopping, providing context for when to use alternatives. However, it does not explicitly contrast with the sibling tool 'codex' or state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codexA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses blocking behavior, git worktree creation for full-access, and return values (final result, session_id, git diff stats). No annotations are present, but description handles behavioral disclosure well without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no redundancy. The primary purpose is front-loaded, and each sentence adds necessary information without superfluous detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Provides adequate overview of behavior and return values. However, several parameters (cwd, topic, prompt) are not fully explained, and the output schema is not detailed. Given the tool's complexity, slightly more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low (20%). The description adds value for 'sandbox' (already described in schema) and 'session_id' (resume context). However, 'prompt', 'cwd', and 'topic' lack explanation, requiring some inference from parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute a Codex task and block until completion,' specifying the verb and resource. It distinguishes from siblings like codex_cancel and codex_dispatch by highlighting synchronous behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use read-only vs full-access sandbox modes ('Use read-only sandbox for reviews/analysis, full-access for code modifications'). Lacks comparison with sibling tools but adequately covers the key parameter choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that with task_id it returns progress events if running or result/diff if completed. No annotations provided, but description covers behavioral traits adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no wasted words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given optional parameter and output schema, description covers all needed context: both modes, what to expect in output (progress/result/diff). No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage, but description fully explains the single optional parameter task_id: its two roles (single vs list), adding crucial meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Check task status' and distinguishes two modes (single task vs list all), making it distinct from siblings like codex_dispatch or codex_cancel.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use each mode: pass task_id for single detail, omit for list. Lacks explicit exclusions but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.