Server Quality Checklist

Tool Scores

• orchestrate_archiveC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden. It states that deliverables are archived and the lock is released, but it does not explain whether archiving is reversible, what happens to in-progress work, how the lock is determined, or what side effects the operation has beyond that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded, two-clause sentence with no filler. Every phrase adds behavioral information, and nothing needs to be removed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has two parameters, no annotations, and unclear side effects, yet the description provides only a high-level action. The agent is left without enough context to safely use 'force' or understand the impact of archiving. An output schema exists, but it does not compensate for the missing behavioral and parameter context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented parameters. It does not mention 'force' or 'workspace_root' at all; the agent cannot infer what 'force' controls or how 'workspace_root' changes behavior. This is a significant gap for a tool with a default force=true.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('archive') and a specific resource ('current orchestration deliverables'), plus a secondary action ('release session lock'). This makes it distinct from sibling lifecycle tools like orchestrate_init or orchestrate_approve, though it does not explicitly name those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when this tool should be used versus ordinary_status, orchestrate_approve, or orchestrate_verify. The name implies it is a finalization step, but the description does not state prerequisites, sequencing, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orchestrate_approveC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosure. While 'Grant human approval' conveys the core action, it does not mention what changes as a result, whether the action is reversible, whether authentication/human-level context is required, or how approval affects the orchestration workflow.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant filler or restatement of the tool name. It is short and easily parsed, which is appropriate for a tool with one optional parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists and the tool has only one optional parameter, this is a workflow-changing approval action in an orchestration toolset. The description does not explain where approval fits in the orchestration lifecycle, what changes after approval, or what happens if workspace_root is omitted/a different workspace is selected, leaving the agent to infer critical context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description never mentions workspace_root or its role. The parameter name and schema type signal that it is an optional workspace path, but the description does not compensate for the missing schema documentation or clarify what happens when it is null versus set.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Grant') and resource ('human approval for the current phase deliverable'), making it distinguishable from siblings like orchestrate_init, orchestrate_verify, and orchestrate_archive. It is clear that this is an approval action rather than a status check or verification, though 'current phase deliverable' is not fully qualified.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to call this tool versus alternatives, such as after orchestrate_verify succeeds or before orchestrate_archive. It also does not state prerequisites for approving the current phase deliverable, so an agent has to infer the workflow context entirely from the tool name and sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orchestrate_get_dag_batchesB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. 'Compute' suggests a read-only calculation, but the description does not explicitly state that it mutates nothing, does not execute tasks, or what happens when plan.md is missing or malformed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, front-loaded with the action verb and resource, with no filler or redundant detail. Every word contributes to the core meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists and the parameter count is low, the description still lacks important context: when to call the tool, how to supply the workspace root, and whether any state changes occur. It provides only the barest outline in what is otherwise a structured orchestration workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is zero parameter coverage in the schema, and the description does not explain the workspace_root parameter or what a null value means. The phrase 'from plan.md tasks' loosely connects to a workspace but does not compensate for the missing parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('compute topological execution batches') and identifies the source data ('plan.md tasks'). It is clearly a batch-planning/read-only computation tool, distinct from siblings like orchestrate_get_agents, though it doesn't explicitly name or differentiate sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies that a plan.md file must already exist and that the result is intended for execution sequencing, but it does not state prerequisites, when not to use it, or how it fits relative to approve/verify/status. The usage context is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orchestrate_initB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It only mentions 'HMAC anti-tamper security' and gives no details about what initialization does, what side effects occur, whether an existing session must be absent, or what happens on failure. This is thin disclosure for a state-creating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one clean front-loaded sentence with no wasted words. It is concise and scannable, though the brevity comes at the cost of missing useful usage and semantic guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema does carry return-shape information, and the two parameters are simple, so the tool is minimally callable. However, the description does not explain the orchestration setup context, how workspace_root should be used, or how this tool fits in the orchestration lifecycle among six sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain either task_description or workspace_root. The parameter names are somewhat self-explanatory, but the tool description itself contributes no meaning, and with a nullable workspace_root, the intended semantics and null behavior remain ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The phrase 'Initialize a new orchestration session' gives a specific verb and resource and clearly distinguishes it from lifecycle siblings like orchestrate_status and orchestrate_archive. The HMAC clause is a security qualifier rather than a purpose statement, so it does not help define intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'Initialize' and the qualifier 'new' imply when this should be used, but the description does not explicitly state when to use it versus alternatives, does not mention lifecycle ordering, and provides no exclusions or prerequisites. Absence of explicit routing leaves some interpretation to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orchestrate_statusB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The word 'Query' implies a read-only operation, which is helpful in a tool with no annotations. However, it does not explain what happens when there is no active session, whether the status is live or cached, or whether calling it has side effects. The description gives the basic safety signal but leaves important behavioral context uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence and waste-free; it states the action and the object immediately. For a simple status-query tool with a single optional parameter, this level of conciseness is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists so the return format is at least partially covered elsewhere, and the tool itself is simple. What is missing is the workflow context: when to call it relative to orchestrate_init or orchestrate_approve, and whether the workspace_root parameter is needed to distinguish statuses. The description is minimally adequate but not fully complete for the orchestration workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the parameter, but it never mentions workspace_root or how it affects the query. The schema supplies only the name, type, default, and a title, leaving the agent uncertain what a null value means and whether the parameter is needed for the current session.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Query') and names the resource ('current session status' and 'active phase'), which makes the tool's core purpose clear and separates it from the action-oriented sibling tools like init, approve, and verify. It is not a 5 because 'active phase' is slightly ambiguous and there is no explicit statement distinguishing it from other status-like queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the sibling tools, no mention of prerequisites like orchestrate_init, and no indication of what conditions call for status checking. An agent must infer usage from the verb and name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orchestrate_verifyB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the key side effect: it advances the phase on success. With no annotations provided, this is important and helpful. However, it does not clarify behavior on failure, whether verification is read-only, whether changes are reversible, or what 'deliverables' includes. It covers the most critical mutation but leaves behavioral details incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence. Every word earns its place, and the core action and conditional outcome are immediately clear. There is no fluff or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists and only one optional parameter is present, the description covers the basic workflow and final state change. But with no annotations and no parameter guidance, the agent is left guessing about the meaning of workspace_root, failure behavior, and boundaries of 'current phase deliverables.' It is adequate but not complete for a tool that mutates phase state.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the sole parameter workspace_root is not mentioned in the description at all. The schema only provides the name, null option, and default, which does not explain its role or whether/how it affects verification. The description fails to compensate for the total lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Run machine verification') on a specific target ('current phase deliverables') and the conditional outcome ('advance phase on success'). This clearly distinguishes orchestrate_verify from siblings like orchestrate_approve, orchestrate_status, and orchestrate_init without needing to inspect schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool — when current phase deliverables need machine verification before advancing — but does not explicitly state when not to use it or call out alternative tools. Sibling tools exist for different purposes, but no direct comparison or exclusion is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orchestrate_get_agentsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and does well by disclosing that names are registered OpenCode subagent IDs, spawnable via the subagent tool even if hidden. It adds meaningful insight beyond 'list agents', though it omits non-critical details like pagination or ordering.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with no filler. The first states the primary purpose, and the second adds high-value operational nuance about spawnability and catalog visibility.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with an output schema present, this description is fully sufficient. It tells the agent what is returned, that names are usable IDs, and that the list may include hidden agents, leaving no essential gap for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is effectively complete, so the baseline of 4 applies. No parameter documentation is needed, and the description correctly avoids inventing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states a specific verb ('List') and resource ('all specialized orchestrator agent personas'). It also differentiates from siblings by explaining these are registered subagent IDs, distinct from tools like orchestrate_init or orchestrate_get_dag_batches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear functional context: the returned names are usable as subagent IDs even when hidden from the catalog, which implies when to use this tool. It does not explicitly name alternatives or exclusion criteria, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.