Server Quality Checklist

Tool Scores

• delegate_cancelA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does real work: it discloses the agent-loop abort and the critical side-effect that background processes survive unless kill_children is set. It stops short of stating reversibility, post-cancel state, or behavior on already-finished jobs, but the essential behavioral surface is exposed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three terse sentences, main action front-loaded, with every sentence earning its place: purpose, abort scope, and the kill_children caveat. No filler or redundant schema restatement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity 2-parameter tool with no output schema, the description covers purpose, the cancellation mechanism, and the conditional side effect. Minor gaps remain – error cases (job not found/already finished), idempotency, and return value – but nothing an agent needs to invoke correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers kill_children at 50%; the description reinforces that same parameter by tying it to the default background-process behavior, adding mild value. job_id is left to the schema and is self-evident as the job identifier, so no meaningful semantics are added beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  'Stop a running delegated job' provides a specific verb and resource, and 'Aborts the agent loop' clarifies the mechanism. The action is clearly distinct from sibling tools (delegate_start, delegate_status, delegate_result, ask), though it doesn't explicitly name an alternative as a top-rating definition might.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: call this when a delegated job is running and should be stopped. There is no explicit when-to-use, prerequisites, or exclusions, and it doesn't route the agent toward a sibling for related needs (e.g., checking state first via deelegate_status).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• askA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden. It is transparent about being one-turn and offline, but it tells agents to use delegate_start for anything needing files, while the input schema offers a files parameter for reading absolute paths as context. This internally contradicts the tool's actual capabilities.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences: the first defines the core behavior, the second scopes use cases and exclusions. Key constraints are front-loaded with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main use cases, exclusions, and the one-tun nature, and schema descriptions cover all parameters. However, the conflict around file handling prevents an agent from confidently deciding whether the files parameter should be used or avoided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so prompt, model and files are already documented. The description adds only that the prompt should be a single self-contained question, but it muddies the files parameter by delegating file-related requests elsewhere.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description names a specific verb and resource: asking Yevgeny a single self-contained question and getting an immediate one-turn answer. It also distinguishes itself from the delegate_* siblings by stating 'No tools, no web access, one turn.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit usage context: summarising, extracting, reformatting, classifying, drafting. It also states when not to use it — for research, files or computation use delegate_start — so an agent has clear routing criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delegate_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It clearly states the operation is non-blocking and enumerates the behavioral scope: state, steps taken, tool calls, background process liveness, and log tail. It does not cover edge cases like invalid job_id or whether the call is read-only, but the wording strongly implies a safe inspection operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with a colon that cleanly introduces a specific list of what the status check returns. Every clause adds useful information, and there is no redundant or filler wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-param status tool with no output schema, the description covers the key invocation context and return content well: state, steps, tool calls, background processes, and log tail. It lacks formal response structure and error behavior, but those are minor for an agent deciding whether and how to call this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%: only log_lines has a description, while job_id is bare. The description gives contextual meaning to job_id via 'a delegated job' and matches log_lines through 'tail of its log', but it does not add detailed parameter semantics beyond what the schema already provides. This is adequate but not compensative for the undocumented job_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Check' and names the resource ('a delegated job') while enumerating exactly what the status check covers: state, step count, tool calls, background process liveness, and log tail. This clearly separates it from sibling tools like delegate_start, delegate_cancel, or delegate_result.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without blocking' implies it is meant for non-blocking async status monitoring, which is useful context. However, it does not explicitly mention when to prefer delegate_status over delegate_result or when to consider delegate_cancel, nor does it state any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delegate_resultA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It adds the non-obvious caveat that a finished job may still have background processes running and directs the agent to delegate_status for those, while the 'collect' wording implies a read-only retrieval. It does not explicitly state that no files are deleted or that the call is non-mutating, but it is considerably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core result and file-listing behavior, then a short caveat and sibling pointer. Every sentence earns its place with no redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, it tells the agent what will be returned: a final report plus a listing of scratch files. It also covers the key ambiguity about background processes. It does not describe error behavior for unknown job_ids or output formatting, but for a focused result-collection tool the essentials are present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers include_files but not job_id. The description's mention of 'delegated job' indirectly clarifies job_id as the identifier of a previously delegated job, but it does not explain include_files beyond the schema's own text. At 50% coverage, it partially compensates but relies on the schema for half the parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states a specific, non-trivial action: 'Collect the final report from a delegated job' plus a second concrete deliverable, 'a listing of every file it wrote to its scratch directory.' This distinguishes it from siblings like delegate_start, delegate_status, and delegate_cancel, which handle creation, status, and cancelling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly routes the agent to delegate_status when it needs background-process info, giving an alternative for a specific situation. It implies the main use case is after a delegated job has finished, though it does not explicitly say 'call only when delegate_status reports done.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• yevgeny_healthA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden. The word 'Check' strongly implies a read-only diagnostic with no destructive side effects, and the list of checks tells the agent what state is inspected. It does not describe the return format or failure behavior, but for a health check this is a reasonable level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose ('Check that Yevgeny is usable') and then uses a colon-separated list to enumerate specific checks. Every word earns its place, with no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless health-check tool with no output schema and no annotations, the description provides sufficient context for an agent to select it and invoke it correctly. It specifies the subject of the check and the concrete dimensions of usability, covering the essential decision-making needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds meaningful context by defining what 'usable' means (Ollama reachable, models pulled, Python interpreter, recent jobs), which helps the agent interpret the tool's purpose even though there is nothing to configure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Check' and names the target resource 'Yevgeny', then enumerates exactly what is verified: Ollama reachability, pulled models, the resolved Python interpreter, and recent jobs. This clearly distinguishes it from the sibling delegation tools, which perform actions rather than diagnostics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is a health-check/diagnostic tool to confirm Yevgeny is usable, but it does not explicitly state when to use it versus alternatives or when not to use it. With siblings like delegate_start and delegate_status, an agent can infer it is a preflight check, but the guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delegate_startA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses asynchronous behavior (returns at once, poll later), support for multi-hour background simulations, and the read-only restriction on root. It stops short of failure modes or concurrency details, but core behavioral traits are well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences. The primary behavior is front-loaded, then capabilities, then invocation protocol, then brief guidance. Every sentence contributes without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explaines the return value (job id) and the rest of the workflow (poll/collect). It covers capabilities, the required brief format, and the root condition. It omits error/health edge cases, but those are plausibly covered by sibling tools lik e yevgeny_health.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful usage meaning beyond the schema: it instructs the caller to provide a thorough brief with goal, sources/directories, and output format, and tells when to omit root. These enrich the task and root parameters beyond their raw schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Hand a multi-step task') and a resource (Yevgeny), and clearly states the immediate outcome (job id back). It distinguishes itself from siblings by explicitly pointing to delegate_status and delegate_result as follow-up tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains when to use the tool (multi-step tasks) and gives clear post-invocation guidance (poll with delegate_status, collect with delegate_result). It also provides a conditional usage detail (omit root for web-only work), though it does not explicitly contrast with delegate_cancel or ask.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.