Server Quality Checklist

Tool Scores

• probe_compactionB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description aligns with the readOnlyHint=true annotation by explicitly stating 'Read-only', and adds the context that it verifies access to owner and runtime state. It does not describe return behavior, error conditions, or what happens if access is denied, but given the annotation already signals safety, the additional context is modest but adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately communicates the read-only nature and primary purpose. It contains no filler or redundancy. While it is terse, it is appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one required parameter, no output schema, and strong annotations, the description covers the basic operation but leaves open important context: how results are reported, why this probe is needed relative to compaction, and what 'owner and runtime state' concretely means. An agent could call it correctly but might not fully understand the expected outcome.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single threadId parameter, and the schema already provides the critical instruction to obtain it from the shell environment and never guess. The tool description adds no further parameter semantics, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('verify') and a specific resource ('owner and runtime state of the specified current Codex VS Code chat'). It clearly distinguishes itself from the sibling compaction tools by positioning this as a read-only verification action rather than status lookup, scheduling, or cancellation. However, 'owner and runtime state' is somewhat vague and could be clearer about what is being probed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus the sibling tools (compaction_status, schedule_compaction, cancel_compaction). It implies a read-only verification role but does not state conditions, prerequisites, or alternatives. The agent must infer the intended usage context from the name and siblings alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compaction_statusA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true, and the description's 'Read' is consistent with that. The description adds genuinely useful behavioral nuance: 'completed requires a new completed contextCompaction event, not just an accepted request,' which prevents an agent from mistaking an accepted request for a completed one. It does not describe the return payload, but the annotations lower the burden for safety-related behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. The core action is front-loaded, and the second sentence adds a compact, high-value behavioral caveat without restating schema or annotation facts.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read-only tool with a readOnlyHint annotation, the description covers the purpose, the 'local' scope, and a key status-interpretation condition. It is sufficient for an agent to invoke correctly, though it could slightly improve by pointing to probe_compaction when a lighter check is needed or by outlining what the result contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter, threadId, is already fully documented in the schema with explicit guidance to obtain it from the shell environment and never guess. The description itself does not add any parameter-specific meaning beyond what the schema provides, so it sits at the baseline for 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Read the local compaction job result.' This clearly communicates the tool's function and read-only nature. However, it does not explicitly distinguish itself from the sibling tool 'probe_compaction,' whose name could also imply checking compaction status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The main clause gives a clear activity, and the caveat about 'completed requires a new completed contextCompaction event' adds conditionality. However, it never explicitly says when to use this tool versus 'probe_compaction,' 'schedule_compaction,' or 'cancel_compaction,' nor does it state when not to use it. Usage context is implied rather than directly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_compactionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate the operation is not read-only and not flagged as destructive, but the description adds important behavioral context: only pending dispatches are affected, and sent requests are irreversible. This goes beyond the structured annotations and helps the agent set accurate expectations about side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences with the core action, the critical caveat, and a follow-up instruction. No filler or redundant restatement; the most important scoping information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a cancellation tool with two required parameters, no output schema, and clear annotations, the description covers the essential behavior, the limitation about already-sent requests, and the recommended next step of inspecting status. It omits failure modes or return-value details, but these are not critical for correct invocation here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documents threadId but leaves jobId without a description, so schema coverage is only 50%. The description compensates slightly by saying 'exact job ID,' which reinforces that jobId must precisely identify the target, but it does not add meaningful semantics for threadId or go beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Cancel') and names a precise resource ('pending compaction/continuation dispatches') while scoping it to an exact job ID. This clearly differentiates it from sibling tools like schedule_compaction, probe_compaction, and compaction_status without needing to infer from the tool name alone.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly signals this tool is for canceling pending dispatches and explicitly warns that already sent requests cannot be undone, which defines the boundary of when it is effective. It also advises inspecting status afterward, which implies a follow-up with a status tool, though it does not explicitly name alternatives or provide a when-not-to-use condition.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_compactionA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description richly discloses behavior beyond the minimal annotations: exactly one compaction is scheduled, handoff is saved verbatim to a per-job file, the schedule cancels on new user input or a stopped turn, the compaction prompt is not overridden, and there are no automatic retries. These are critical operational details the agent could not infer from annotations alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the primary action and authorization requirement appear first, followed by handoff/resume semantics, cancellation behavior, and status-check guidance. Every sentence carries operational meaning with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a scheduling tool with no output schema, the description covers the essential operational contract: when to call it, what it schedules, how handoff works, resumption semantics, cancellation, status checking, and absence of retries. The sibling tools for probing/status/cancellation fill the remaining workflow context. Nothing critical appears missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 50% schema coverage, the description compensates meaningfully by explaining handoff persistence, the one-turn resume behavior, and the fact that the compaction prompt is not overridden. The schema already documents threadId and handoff fields well, so the description adds selective, high-value context rather than repeating schema text. It does not fully elaborate on all parameter edge cases, hence not a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Schedule ONE compaction after this answer.' It clearly differentiates from sibling tools (probe_compaction, compaction_status, cancel_compaction) by focusing on scheduling rather than probing, checking, or canceling. The scope and constraints are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states the key precondition: 'Only when the user authorizes compaction.' It also restricts resume:true to authorized continuation and notes cancellation conditions. It does not explicitly name sibling tools or provide when-not-to-use comparisons, but the context is strong enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.