Server Quality Checklist

Tool Scores

• doctorA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the description only needs to add context beyond safety. It adds useful behavior: workspace defaults to server cwd and deep enables no-quota help plus model-catalog checks. It does not describe the output format or clarify ambiguous terms like 'recursion', but the safety profile is already covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler: the first front-loads the diagnostic targets and workspace default, the second covers the 'deep' flag concisely. The phrasing is slightly cryptic, especially 'no-quota help', which prevents a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should clarify what a diagnosis produces or returns, but it does not. Both parameters and the safe read-only behavior are addressed, yet ambiguous terms such as 'recursion' and 'no-quota help' and the lack of any mention of the tool's output leave meaningful gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for documenting the parameters. It explains 'deep' as adding no-quota help and model-catalog checks, and it gives the workspace default as server cwd. It stops short of specifying workspace format or the exact meaning of 'no-quota help', but both parameters receive meaningful semantic context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb 'Diagnose' and names concrete targets: plugin/CLI/login, recursion, and workspace. It also distinguishes the 'deep' mode by what it adds. It does not explicitly connect to the annotation title's 'Codex delegation setup' or differentiate from sibling tools, so it is not a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The verb 'Diagnose' and the list of target areas imply when the tool should be used, and 'deep adds no-quota help and model-catalog checks' gives a condition for enabling extra checks. However, it does not explicitly say when to use doctor versus the sibling tools delegate/cancel.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delegateA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  It discloses a non-obvious behavioral quirk: 'a run that spawns then fails returns normally.' This adds valuable operational context beyond the annotations, which only flag destructiveHint and openWorldHint. It does not contradict any annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sentences, each earning its place: purpose, shell prohibition, failure-mode warning, default guidance, and pointer to the skill. No redundant or filler wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 10-parameter mutation tool with no output schema, the description includes the most critical operational warning and points to the delegate skill for further detail. It does not explain status-check mechanics, but the schema and skill reference compensate reasonably.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all parameters well. The description adds default-keeping guidance for model, reasoningEffort, and fast, but does not deeply expand on parameter semantics. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Delegate a coding task to OpenAI Codex.' This clearly identifies what the tool does. It does not explicitly name the sibling tools (cancel, doctor), but the verb+resource combination is distinct enough that an agent can tell it apart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs agents to use this tool instead of running codex from the shell, provides a clear verification step ('Check status before trusting result'), and advises keeping model, reasoningEffort, and fast at defaults unless the user asks. This is strong, actionable guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancelA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and idempotentHint=true, and the description adds meaningful behavioral context: it waits for settlement, returns even if a process tree survives the kill deadline, and warns via the delegate result. It also enumerates the three possible statuses. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences with no filler. The most important selection semantics are front-loaded, and the status outcomes are compactly listed at the end. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a cancellation tool with no output schema, the description covers selection semantics, wait behavior, failure behavior, and status outcomes. It does not describe the exact shape of the return value, but the status list and warning note give an agent enough to interpret the result. The sibling tools are simple enough that no further context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must carry the parameter semantics, and it does: delegationId selects one run, threadId selects all on a thread, and neither selects all. It also explains the special significance of delegationId as the only handle for a wedged startup. This fully compensates for the empty schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Cancel active runs') and immediately distinguishes the two selection modes (delegationId vs threadId) and the all-runs fallback. It also names the sibling context by explaining when delegationId is the only handle, which clearly separates it from delegate and doctor.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use delegationId (announced in progress before spawn, the only handle if a run wedges at startup), when to use threadId (all runs on its thread), and when to use neither (all active runs). This is direct when-to-use guidance with no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.