Server Quality Checklist

Tool Scores

• codexB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It only says 'synchronously', omitting details like blocking, duration, side effects (sandbox, worktree creation), or return value. This is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently states the core purpose. It is appropriately concise, though could be slightly more specific.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, no output schema, and no annotations, the description is too minimal. It fails to explain what a Codex session does, what it returns, or important behaviors (e.g., it may take time, creates worktrees).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds no additional meaning beyond what the schema already provides. The phrase 'Parameters match the official Codex MCP tool' is generic and not helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it runs a Codex session synchronously, which is a specific verb+resource. It implicitly differentiates from siblings like codex_async by mentioning 'synchronously', but does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit when-to-use guidance. It implies synchronous execution, but does not contrast with async or other management tools. Agents are left to infer usage from the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_replyA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It mentions using 'codex exec resume' under the hood, but does not disclose whether the tool is blocking, error handling, or rate limits. This is minimal behavioral disclosure for a continuation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus a brief technical note. It is front-loaded with the core purpose, no wasted words, and efficient for agent consumption.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required params, no output schema), the description is minimally viable. It explains the action and underlying command but lacks usage guidelines and behavioral details. Could be more complete with usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no new meaning beyond restating the parameter roles. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: 'Continue a Codex conversation' by providing a thread ID and prompt. It distinguishes from sibling tools like 'codex' (which likely starts new conversations) and async variants, making it easy for an agent to select appropriately.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when continuing an existing conversation, and the sibling list provides context. However, it lacks explicit 'when to use' vs. alternatives, such as preferring 'codex' for new sessions. Still, the guidance is clear for most scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_cancelA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description carries full burden. It discloses that the process is terminated, task marked cancelled, and worktree/partial output preserved. This is good but lacks mentions of permissions or irreversible effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the action. Every word serves a purpose; no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema), the description provides sufficient behavioral and side-effect information to understand the tool's function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description adds minimal value beyond the schema description. It basically restates the parameter's purpose without additional details on format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action (kill/cancel) and resource (async Codex task). It distinguishes from siblings like codex_async (starts tasks) and codex_status (checks status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage for canceling a running task but provides no explicit guidance on when to use versus alternatives or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the tool is for live status (no side effects) and specifies what information it provides. It could mention polling behavior or rate limits, but the transparency is adequate for a read-only status check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the main action, and contains no fluff. Every sentence provides useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 param, no output schema). The description covers functionality and output details. It omits the exact output format, but given no output schema, this is acceptable. Overall complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for task_ids. The description does not add extra meaning beyond the schema, but baseline 3 is appropriate since the schema already documents the parameter adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves live status of async Codex tasks, specifying what information it shows (last tool call, reasoning, progress). It differentiates from siblings like codex_async (starts tasks) and codex_cancel (cancels them).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after starting an async task with codex_async and notes it works on both running and completed tasks. It does not explicitly exclude use cases or state alternatives, but context from sibling names provides enough guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_asyncA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It transparently explains the async behavior, immediate return, and worktree isolation. However, it does not disclose potential side effects, authorization needs, or task persistence, which are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with three sentences, each adding distinct value. The main action is front-loaded, and every sentence earns its place without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 parameters, async workflow), the description covers the key workflow: start, get task_id, use companion tools. It lacks details on return format and error conditions, but is otherwise complete for a background task tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by explaining the worktree parameter's behavior and purpose, but does not enhance understanding of other parameters beyond their schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool starts a Codex task in the background and returns a task_id immediately. It distinguishes itself from siblings by referencing codex_wait and codex_status for collecting results and monitoring progress.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly guides when to use: for background task execution. Mentions alternatives (codex_wait, codex_status) and provides context for using the worktree parameter to avoid conflicts in parallel tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_waitA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It clearly states the tool blocks, has a default timeout of 1800s, and that timed-out tasks continue running. This covers the key behaviors without omitting critical details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences plus a timeout note. It immediately states the purpose and provides necessary usage context without superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and the tool's simplicity (two parameters), the description covers all essential aspects: purpose, when to use, timeout behavior, and follow-up options. It is complete for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for parameters is 100%, with descriptions already present. The tool description adds context about default timeout and non-killing behavior, but this is also captured in the schema's timeout description. Thus, parameter semantics are adequate but not significantly enhanced beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Block until one or more async Codex tasks complete, then return all results.' It distinguishes from siblings like codex_status and codex_async by noting that it avoids polling and contrasts with launching tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises to 'call once after launching codex_async tasks' and explains timeout behavior: 'If a task times out, it is NOT killed -- it keeps running. You can call codex_wait again... or use codex_status to check progress.' This provides clear guidance on when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.