Server Quality Checklist

Tool Scores

• agent_compareA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions READ-ONLY mode and independent failure, but does not describe output format or potential side effects. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all essential. No fluff. Efficiently covers purpose and behavioral notes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Missing output schema, so description should clarify return format. It says 'return both results verbatim' but does not specify structure. Also lacks detail on directory constraints. Adequate but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description adds no new parameter meanings beyond what schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it runs both Codex and OpenCode against the same prompt in READ-ONLY mode and returns results verbatim. Distinguishes from sibling tools like codex_run and opencode_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says the caller synthesizes the conclusion and that one agent failing does not suppress the other. Provides good usage context, though could explicitly mention when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codex_runA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behaviors: local installation requirement, non-interactive mode, sandbox, default read-only, and return structure (final message plus summaries). Warns about enabling writes via workspace_write.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three succinct sentences, each providing distinct value: purpose, usage guideline, and return description. No redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and no output schema, the description covers the tool's purpose, key behavioral constraints, and output summary. Could briefly elaborate on 'events' output mode for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% parameter description coverage, so baseline is 3. The description adds context for the mode parameter (when to use workspace_write) and hints at return format, but does not significantly enhance understanding of other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly identifies the tool as running Codex CLI non-interactively in a sandbox, with a default read-only mode. However, it does not explicitly differentiate from sibling tools like opencode_run, limiting clarity slightly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use workspace_write only when writes are enabled server-side.' This helps the agent decide when to use each mode. Missing contrast with opencode_run or other alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_healthA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It thoroughly lists what is reported, including 'whether writes are permitted' and 'current concurrency usage,' which implies a safe read operation. It could mention that it has no side effects, but the description is sufficiently transparent for the given complexity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose. Every word contributes meaning, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description fully covers what the tool does. It lists all key aspects of the agent environment, providing complete contextual information for an agent to decide when to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100%. The description adds meaning by detailing what the output includes, making it more valuable than the empty schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs and nouns: 'Report the local agent environment:' followed by a detailed list of items (Node version, Git availability, etc.). This clearly distinguishes it from siblings like codex_run and opencode_run, which perform execution tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for health checks but does not explicitly state when to use this tool vs alternatives or provide exclusions. An agent can infer but lacks explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opencode_runA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses sandboxed directory, non-interactive execution, side-effecting actions with auto_approve, and session continuation. However, it omits details like idempotency or error handling, which would be useful for risky operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy. The main action is front-loaded, followed by essential details. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters (2 required) and no output schema, the description covers the core behavioral contracts and parameter meanings. It could mention what the tool returns (e.g., output of CLI) but is otherwise adequate for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds value beyond schema by explaining that auto_approve enables side effects and requires server-side permission. It also contextualizes session_id as 'session continuation'. This reduces ambiguity for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb-resource pairing ('Run the ... OpenCode CLI') with context ('non-interactively in a sandboxed working directory'). It clearly differentiates from siblings like 'agent_health' (health check) and 'codex_run' (likely another CLI runner) by specifying the exact CLI and mode.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use auto_approve and mentions session continuation, but does not explicitly state when to prefer this tool over siblings or when not to use it. Still, the context is clear for an experienced user.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.