Server Quality Checklist

Tool Scores

• codex_listA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only states it lists jobs with status and summaries, but lacks details on pagination, ordering (though limit param says 'most recent first'), rate limits, or side effects. The description is too minimal for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no redundant words, front-loaded with the core action. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (status and prompt summaries mentioned), the description is somewhat complete for a simple list operation. However, it lacks details on error handling, empty results, or additional behavioral context that would fully inform an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'limit', and the schema itself provides a description including default and ordering. The tool description adds no extra meaning beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool lists recent codex jobs, including status and prompt summaries. The verb 'list' and resource 'recent codex jobs' are specific and distinguish from sibling tools (cancel, poll, start) which are different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use or avoid this tool. The purpose is implied by the name and description, but no alternatives or exclusions are mentioned. Siblings have distinct purposes, so usage is inferred but not clarified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_pollA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states the purpose but lacks details on behavioral traits such as whether the tool is idempotent or safe to call repeatedly. Since annotations are absent, the description carries the burden, and it only provides minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no extraneous words. It efficiently conveys the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description does not need to explain return values. It covers the essential purpose and scope, though it could mention that the tool can be called multiple times safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds no additional meaning beyond what the schema already provides for the two parameters. The description's mention of 'output' hints at tail_lines, but this is redundant with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Poll' and the resource 'status and output of a running (or finished) codex job', which is specific and distinguishes it from sibling tools like codex_start, codex_cancel, and codex_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after a job is started, but does not explicitly provide when-not-to-use or alternatives. The context is clear enough for an agent to infer appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_cancelA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that cancellation is done by sending SIGTERM to the subprocess, which is a key behavioral trait. It does not cover edge cases like job already finished, but the main behavior is well communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 12-word sentence, front-loaded with the action, and contains no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, single action), the description is nearly complete. It could mention what happens if the job is not running, but the output schema likely handles error responses.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes the only parameter (job_id) with 100% coverage. The description adds no new semantic information beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Cancel') and resource ('running codex job'), and clearly distinguishes from siblings like codex_list, codex_poll, and codex_start.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly states when to use (to cancel a running job with a job_id from codex_start) but does not explicitly state when not to use or provide alternative scenarios. However, the simplicity of the action mitigates the need for extensive guidelines.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_startA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses async, non-blocking, immediate return of job_id. Does not mention side effects or auth, but core behavior is adequately covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Front-loaded with purpose and key behavior. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Has output schema. Describes async nature and returns job_id. Could mention cancellation via sibling codex_cancel, but sufficient for a simple start tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline 3. Description does not add meaning beyond schema; each parameter is defined in schema. No extra context provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Start', resource 'codex task', and key behavior 'asynchronously in the background'. Distinguishes from siblings by mentioning that codex_poll is used to check progress.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance to use codex_poll for progress checking. Implicitly tells when to use this tool (async tasks) but lacks explicit when-not-to-use or alternatives beyond polling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.