Server Quality Checklist

Tool Scores

• handoff_jobsB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral context. It indicates a read operation by saying 'List', which is helpful, but lacks details on pagination, ordering, authentication needs, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence that is front-loaded with the action and resource, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lists the returned fields, which is good, but lacks details on ordering, pagination, error conditions, or any additional behavior. Given the tool has an output schema and one optional parameter, the description is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'limit' has no description in the schema (0% coverage) and the description does not mention it or provide any guidance on its usage, format, or effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and a clear resource 'recent handoff jobs', and explicitly mentions the fields returned (id, backend, state, elapsed, prompt preview). This distinguishes it from sibling tools like handoff (likely create), handoff_status, handoff_backends, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not provide context about when not to use it, prerequisites, or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handoff_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully bears the burden of behavioral transparency. It details the polling nature, return fields, and the condition to stop polling. It does not disclose potential errors like invalid job_id, but overall is transparent about typical behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively concise, using a short paragraph and a bullet-like list for return fields. It front-loads the purpose and then details parameters and return. Could be slightly more structured but still efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description adequately covers the polling pattern, return fields, and parameter usage. It is complete enough for an agent to use correctly without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. It explains job_id as from handoff_start and tail as lines of progress log. This provides context beyond the schema's type-only definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it polls a background handoff job for state, elapsed time, live progress, and result. It distinguishes from siblings like handoff_start by focusing on polling, though it doesn't explicitly contrast with handoff_jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description instructs to call repeatedly until state != 'running' and specifies that job_id comes from handoff_start. It provides clear context for when to use, though it doesn't explicitly list when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handoff_startA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits: it returns immediately, runs as a background job, has a default 30-minute timeout, and returns a job id. With no annotations provided, the description carries full burden and covers the main async behavior well. However, it does not mention error handling or failure scenarios, which would enhance transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph of four sentences. The first sentence states the core purpose, followed by usage guidance and return details. Every sentence is informative and without fluff, making it appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description lacks detailed parameter documentation for the 6 parameters. Given zero schema descriptions and high parameter count, the description fails to provide complete context. It does not reference sibling tools like handoff_backends for additional context, leaving gaps for an agent trying to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the 6 parameters. It only states 'Args mirror handoff' without explaining any parameter specifics (e.g., what 'approve' does, valid 'backend' values). This adds minimal value beyond the schema, leaving the agent without necessary guidance on parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Start a handoff as a background job and return a job id immediately,' using a specific verb and resource. It distinguishes the tool from sibling tools like handoff (presumably synchronous) and handoff_status (polling). The purpose is unambiguous and well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using this tool for long or agentic handoffs and directs to poll handoff_status for progress. It contrasts the async path with a synchronous alternative ('instead of blocking on one long call'), providing clear when-to-use guidance and an explicit alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handoff_guidanceA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool returns a guide, but does not mention any side effects, rate limits, or other behavioral traits. Since it is a read-only tool, the description is adequate but could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a maintainer note. The core information is concise, but the note to maintainer is extraneous for an agent. Could be slightly more streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and has an output schema. The description fully explains what the tool returns and why it's useful, making it complete for an agent to understand its purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (0 params), and schema coverage is 100% trivially. Baseline is 4, and the description adds no param info because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a model routing guide specifying which backend is best for which task and when to hand off aggressively. It distinguishes itself from siblings that perform handoffs or check status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining routing guidance, and sibling tools have distinct actions (handoff, status, etc.), making it clear when to use this tool. However, it does not explicitly state when not to use it or provide alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handoff_backendsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description discloses output includes a 'staleness' banner and reminds to refresh guidance. Adequately informs agent of non-obvious behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no redundancy. First sentence gives core purpose, second gives usage trigger, third adds behavioral note. Efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and presence of output schema (mentioned in context), description covers all necessary info: what is reported, when to use, and behavioral note.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so baseline 4 applies. Schema coverage 100% trivially.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States specific verb 'Report' with concrete resources: install, auth readiness, quota pool, freshness note. Distinguishes from siblings by being first call on errors.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use: 'Call this first if a handoff returns an auth/not-found error.' Provides clear context and condition.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handoffA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description covers blocking behavior, temp directory usage, no originals modifications without approval, and return value format. It also clarifies the peer agent has no conversation context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with clear sections but slightly verbose. Front-loads purpose and main guidelines. Each sentence adds value, though could be trimmed for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, usage, parameters, behavioral effects, and return value. References sibling tools. However, could include example of error handling or more detail on output schema structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description explains all 7 parameters with meaningful details (e.g., file type support per backend, default behaviors for approve, timeout meaning). Adds value beyond schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Hand a task off to a peer coding-agent CLI' and distinguishes from sibling tools like handoff_start by noting blocking vs non-blocking. It also specifies when to use each sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this for tasks that finish in a few minutes. For long/agentic work, prefer handoff_start + handoff_status'. Also provides guidance on backend selection and aggressive handoff when usage limit is close.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.