Server Quality Checklist

Tool Scores

• launch-classic-agentA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate this is not read-only, not idempotent, and not destructive. The description adds meaningful behavioral context beyond annotations: the agent 'keeps context across the session' and monitors `task:///all` for status updates. This informs the caller about session persistence and operational behavior. It does not mention rate limits or explicit auth requirements, but the adding context justifies a 4.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized: first sentence states purpose, then behavior and status-monitoring details. It is a bit long but every sentence adds value, and the status section is front-loaded with actionable monitoring instructions. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description could mention return values, but this is a launch tool and the monitoring instructions (task:///all) effectively tell the caller how to obtain results. The description covers the task lifecycle and operational context well, and the 6-parameter schema is fully described. Minor gap: no explicit statement of what the launch call returns, but the monitoring info compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all six parameters are already documented in the schema. The tool description itself does not add parameter-level detail (e.g., format constraints or typical defaults) but the schema already handles this. A baseline 3 is appropriate since the description provides no extra value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool launches a 'general-purpose autonomous agent' and explicitly enumerates example task types (documentation, analysis, file organization) and contrasts with specialized siblings (coder/planner/tester/researcher). This provides a specific verb+resource and distinguishes it from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'for non-specialized tasks' and 'any automation that doesn't fit coder/planner/tester/researcher.' It also adds behavioral guidance ('Investigate first') and notes the tool is 'good for iterative or exploratory work,' which helps the agent choose between this and specialized siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• answer-agentA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool modifies agent state by resuming execution, complementing the annotations which are all false. It explains the effect (resume execution) but doesn't detail side effects or error behavior, which would warrant a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bolded sections, examples, and front-loaded purpose. It's longer but each section earns its place; slightly verbose but not excessive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully explains when and how to use the tool, including finding pending questions and formatting answers. However, it omits any indication of the tool's return value or error handling (no output schema), leaving a notable gap in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptive parameter descriptions, but the tool description adds valuable examples for the answer and answers fields, including the 'OTHER:' prefix and the flow-specific formatting. This goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Submit an answer to a pending question from an agent' and clarifies it resumes execution. This distinguishes it from sibling launch/message/cancel-agent tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear 'When to call' guidance: tasks with status waiting_answer, and differentiates between single-question (Copilot/Claude) and multi-question (Codex) flows. Also explains how to find pending questions via task:///all.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel-agentA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (destructiveHint, idempotentHint), the description discloses the specific kill signal sequence (SIGTERM → SIGKILL), what happens to terminal agents (removed from memory), and the effect of clearing all state. This adds meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bold section labels, making it scannable. It includes only necessary details and examples, though it is slightly longer than the minimum viable description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a cancellation tool with two parameters and no output schema, the description is fully complete. It covers all operational modes, side effects, and auxiliary information (how to find task IDs), leaving no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers both parameters thoroughly, but the description enriches this with concrete JSON examples and clarifies the conditional requirement for 'clear' when task_id='all'. This goes beyond the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with specific verbs and resources: 'Cancel running agents or clear the entire workspace.' It distinguishes itself from sibling tools (launch, message, answer) by focusing on cancellation and cleanup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides detailed instructions for both cancellation modes (specific task IDs vs. clear all) and explains how to find task IDs. It implies the appropriate context for use (when you need to stop agents or clean up), though it does not explicitly name alternative tools to avoid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch-super-plannerA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses behavioral traits beyond annotations: 'Always uses claude-opus-4.6 regardless of model parameter', output location '.agent-workspace/plans/[topic]/', and asynchronous monitoring via 'Read task:///all every ~30s'. The annotations (readOnlyHint=false, openWorldHint=true) are consistent with these details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bolded sections (Workflow, Status) and front-loaded purpose. It is dense but each sentence contributes unique information, though the status list and detailed handoff instructions make it longer than strictly necessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with no output schema, the description covers the full lifecycle: planning, workspace output, handoff to launch-super-coder, and status monitoring. It implies the return of a task ID via 'read task:///{id}' but does not explicitly state the return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions the model parameter only to confirm it is overridden, duplicating the schema's existing note rather than adding new meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with 'Launch an autonomous planning agent' and explains it 'Designs architecture and creates implementation plans as .md files', providing a specific verb+resource. The workflow diagram 'researcher → PLANNER → coder → tester' and mention of launch-super-coder distinguish it clearly from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Use this for any non-trivial task' with a concrete heuristic ('touches 3+ files or has ambiguous requirements, plan first'). It also directs the user to pass the generated .md files to launch-super-coder, clarifying the tool's role in the pipeline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch-super-researcherA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate openWorldHint=true and destructiveHint=false; the description adds valuable behavioral context about output location ('.agent-workspace/researches/[topic]/'), the need to read task:///{id} after completion, the don't-cherry-pick handoff rule, and monitoring via task:///all every ~30s. Minor deduction because it doesn't explicitly warn about long-running behavior or costs, but it's richer than most.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose, then a clear workflow, output location, and status loop. Each sentence earns its place. Slight deduction for the bolded emphasis and the somewhat dense status list, but it remains well-structured and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description compensates by explaining what output is produced (.md files), where it goes, and how to retrieve it (task:///{id}). It also covers the monitor loop and handoff. It doesn't detail failure modes or cancellation, but for a launch tool with rich schema and annotations, this is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining the overall workflow purpose of the prompt parameter's handoff target and the don't-filter context_files rule. It could add more detail on how depends_on interacts with context_files, but it does clarify the execution-order-only semantics, which adds meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Launch an autonomous research agent' with specific investigative scope ('Investigates codebases, APIs, libraries, and technical topics') and output type ('.md research documents'). It distinguishes itself from sibling tools by explicitly placing it first in the workflow (RESEARCHER → planner → coder → tester) and emphasizing 'Investigate before you solve'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'use this before planning or coding when the problem space is unclear.' It names sibling tools in the workflow context (planner, coder, tester) and specifies the handoff protocol, so the agent knows exactly when to invoke this versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch-super-coderA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits beyond the annotations: complete isolation ('prompt + context_files are its ONLY context'), the hard failure if non-.md files are passed, the fact that it writes HANDOFF.md notes, and the statuses visible via task:///all. These details enrich the Open World and non-read-only annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than ideal but well-structured with bold section headers and line breaks. It front-loads the core purpose and then provides essential operational details. Each sentence carries meaning, from the isolation warning to the handoff file path, though a few phrases (e.g., 'context_files are MANDATORY') are repeated in spirit elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is highly complete for a complex tool: it covers prerequisites, file constraints, workflow ordering, handoff artifacts, and status monitoring via task:///all. However, there is no output schema and the direct return value of the launch call is never explicitly stated (e.g., a task ID), though the task:///all reference implies it. The agent can infer the immediate response, but a small gap remains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all six parameters, with a description coverage of 100%, so the baseline is 3. The description adds extra semantic value by emphasizing that context_files are mandatory and must be .md only, by telling the agent to pass ALL .md files rather than cherry-picking, and by clarifying the dependency behavior (execution order only) in the workflow narrative.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Launch an autonomous coding agent for implementation, bug fixes, and refactoring.' It also clearly positions the tool as the final implementation stage in the researcher → planner → CODER → tester workflow, distinguishing it from sibling tools like launch-super-planner and launch-super-tester.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: it should be used after the planner completes, and the agent is told to read `task:///{id}` and pass all .md files from the planner's workspace. It also states that non-trivial tasks require investigation and planning first, and that context_files must be created via launch-super-planner or self-authored, thus indicating the proper sequencing relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch-super-testerA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond the annotations: 'Runs in COMPLETE ISOLATION,' 'Read task:///all every ~30s,' and that context_files are 'injected directly into the agent prompt.' These traits are not captured by readOnlyHint, openWorldHint, or destructiveHint, so the description meaningfully discloses how the tool behaves at runtime.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bolded sections (intro, context_files, Workflow, Status) and is front-loaded with purpose. It is moderately sized and every sentence serves a role, though it slightly repeats schema details (e.g., context_files being required) which is redundant but not wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 6 parameters and no output schema, the description covers purpose, isolation, workflow placement, and status monitoring. It does not explicitly state the return value or task ID, but the task:///all reference implies task-based outputs, leaving only a minor gap in explaining what the caller receives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description enriches parameter meaning by emphasizing 'context_files are MANDATORY,' recommending 'Pass ALL files from the coder's .agent-workspace/ — especially HANDOFF.md,' and clarifying the order of depends_on. This practical guidance goes beyond the schema's field-level descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific verb and resource: 'Launch an autonomous testing agent.' It then details the exact scope (Playwright E2E, API testing with curl + jq, running test suites, verification) and positions the tool in a workflow (researcher → planner → coder → TESTER), distinguishing it from sibling tools like launch-super-coder and launch-super-planner.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: 'Chain with depends_on after coder' and states that context_files from the coder's workspace are mandatory, especially HANDOFF.md. It also explains the monitoring loop with task:///all, providing concrete operational guidance. While it does not name alternatives, the workflow placement is an explicit usage directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• message-agentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses a key behavioral trait beyond annotations: it returns a NEW task_id while the original task remains terminal, and instructs monitoring the new ID. This is valuable context that annotations (readOnlyHint, idempotentHint) do not fully convey. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear bold labels (Returns, When to call, Find task_id). Every section adds useful information with no fluff, making it easy to scan while remaining informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no output schema, and rich annotations, the description covers all essential aspects: purpose, when to use, how to locate task_id, and return behavior. It does not need to explain cwd/timeout since those are fully documented in the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaning by explaining that the task_id should be picked from terminal tasks and that the default message 'continue' resumes exactly where it left off, which supplements the schema's parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair: 'Send a follow-up message to an existing agent session' and clarifies that it resumes the session. This clearly distinguishes it from sibling tools like launch-super-coder (new agent) and cancel-agent (cancel).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'When to call' section, specifying terminal agent states (completed/failed/rate-limited) and follow-up instructions. It does not explicitly state when not to call or name alternatives, but the context is clear enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.