Server Quality Checklist

Tool Scores

• delegate_taskA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the behavioral burden. It discloses the side-effect of transcript capture, the verbatim return behavior, and backend/model override semantics. It does not discuss latency, cost, or authentication, but those are not critical for selecting or invoking this tool correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized with a front-loaded summary followed by a clear Args block. Every parameter is explained in one or two lines, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-shot delegation tool, the description covers purpose, parameter semantics, backend resolution, and the return behavior. With an output schema present and sibling context available, no critical invocation detail is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description fully documents all five parameters, including the relationship between backend and model, overriding behavior, and the opt-in nature of capture_transcript. This completely compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Delegate a single-shot task to a configured OpenAI-compatible model' and 'return its text response verbatim.' The 'single-shot' qualifier distinguishes it from the sibling delegate_agentic_task, though it does not explicitly name that sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives concrete guidance on when to use capture_transcript ('when comparing models, e.g. a bake-off') and when not ('rather than for routine use'), and explains backend selection versus DELEGATE_* env vars. It does not explicitly describe when to choose delegate_task over delegate_agentic_task, but context signals and the 'single-shot' phrasing provide reasonable guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_delegationsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden. It discloses the read-only nature (without re-running), sorting (most recent first), truncation of results, and conditional cost reporting. It does not mention pagination or error behavior, but for a simple read-only listing tool these are minor omissions; the disclosed traits exceed typical descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately concise, listing the returned fields in a parenthetical that is useful but slightly dense. The core purpose is stated upfront, and the parameter doc is separated. It could be tightened by moving the field list to a separate line, but it remains efficient and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one optional parameter, no annotations, and an output schema (not provided). The description covers the return semantics (fields, ordering, truncation, cost condition) and the read-only intent. Given the simplicity, nothing essential for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the single parameter 'limit'. It does so explicitly: 'Max number of records to return (default 20).' This adds full semantic meaning beyond the bare schema field, making the tool usable without additional inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists recent delegate_task / delegate_agentic_task calls, enumerates the returned fields (backend, model, task, duration, iterations, success, token usage, USD cost, truncated result), and specifies ordering (most recent first). It also states the intended purpose—answering what a delegated model actually did—which distinguishes it from sibling tools that create delegations or fetch full transcripts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case for inspecting prior delegations without re-running them, but it does not explicitly contrast with siblings like get_delegation_transcript or delegate_task. It lacks explicit when-not-to-use guidance, though the mention of 'without re-running anything' strongly suggests a read-only inspection context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delegate_agentic_taskA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the behavioral disclosure burden. It clearly states that the delegated model gets unattended read/write/execute access within working_dir, that only the final answer is returned, that there are termination conditions, and that transcript capture is opt-in. This is comprehensive and honest about side effects and limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Despite being long, the description is tightly structured: a core behavior paragraph, a safety warning, then a bulleted Args list. Every sentence earns its place, and the most important info (what it does, termination, permissions) is front-loaded. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex delegation tool with 7 parameters, no annotations, and a dangerous access profile, the description covers all critical aspects: scope, termination, access level, return value, optional transcript capture, and backend override. The existence of an output schema is acknowledged but not required to detailed since it says returns only the final answer. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description is the only source of parameter meaning. It explains every parameter in the Args block, including the nuanced interplay between model and backend (backend as a base_url/model/api_key bundle, and that `model` overrides within that backend). This fully compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb and resource: 'Delegate a multi-step task to a model with its own tool-use loop...'. It clearly states the operation's scope (working_dir) and distinguishes itself from tools like get_delegation_transcript by explaining that it returns only the final answer, not the full transcript. This is a specific, unambiguous definition that lets an agent know exactly what it does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the conditions under which the delegated model stops (no more tool calls, max_iterations, timeout_seconds) and warns about unattended file/bash access. It also suggests capture_transcript for comparison scenarios, indirectly routing to get_delegation_transcript. However, it does not explicitly contrast with delegate_task or state when to choose this tool over that sibling, leaving some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_delegation_transcriptA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the error condition for missing transcripts, which is the key behavioral nuance. It does not explicitly state read-only semantics, but that is reasonably implied for a retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two clear sentences and a brief args section. No redundant or filler content; it efficiently conveys all necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is an output schema (as indicated in context), the description need not explain return formats. It covers the essential context: the source of the id, the capture condition, and error behavior. This makes it complete for a single-parameter retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The parameter delegation_id is explained beyond the schema: it is the id from a list_recent_delegations row. This provides actionable meaning on how to obtain the correct value, enhancing the bare integer type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full transcript for a delegation, using a specific verb ('get') and resource ('transcript'). It is distinct from siblings (list_recent_delegations lists, delegate_task delegates), so no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes the precondition (capture_transcript=True), the error behavior when no transcript exists, and instructs to obtain the delegation_id from list_recent_delegations. This gives clear when-to-use guidance and differentiates it from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.