Server Quality Checklist

Tool Scores

• bernstein_post_artifactC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It adds only 'Host effects: makes network requests,' which is marginal because 'post' already implies a network request. It does not disclose versioning behavior, task state changes, idempotency, or failure semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler, and the core action is front-loaded. However, the 'Host effects' sentence is boilerplate and adds little information, preventing a top score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool with 15 parameters, conditional requirements, and four artifact_type variants, yet the description provides almost none of that context. Even though an output schema exists and return-value documentation is not required, the missing parameter semantics and usage guidance leave the description incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for 15 parameters, but it explains none of them. It does not clarify artifact_type variants, required conditional fields, link_kind values, or the meaning of key, poster, target, or invocation_argv_hash.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action and resource: 'Post a versioned artifact to a task on the Bernstein server.' This is a specific verb plus resource and is distinguishable from siblings like bernstein_post_message by the artifact focus, but it does not explicitly differentiate itself from any sibling or mention the artifact_type variants.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus bernstein_post_message or other artifact-related operations. The phrase 'versioned artifact' implies a niche, but no context, exclusions, or alternative tool referrals are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_post_messageB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the host effect 'makes network requests', which is a useful side-effect warning in the absence of annotations. However, it does not elaborate on other behavioral traits such as idempotency, required task state, or error outcomes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with the primary action front-loaded. Every phrase earns its place, making it extremely concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The purpose is clear, but the description lacks usage guidelines, parameter semantics, and differentiation from sibling tools. Given the low schema coverage and absence of annotations, more context is needed for an agent to invoke this tool appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the five parameters. While the schema includes constraints and an enum for 'kind', the description itself adds no meaning beyond the field names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Post') and identifies the resource ('progress message to a task mailbox on the Bernstein server'), clearly distinguishing it from sibling tools like bernstein_post_artifact. It is concise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description only states the action without indicating prerequisites, exclusions, or comparisons to sibling tools such as bernstein_post_artifact or bernstein_complete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_run_statusA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and succeeds by stating it reads the local run journal and audit evidence without changing them, plus host effects: reads files. This gives a clear safety profile of a read-only operation. It does not add details on errors or return format, but the output schema likely covers that, so the provided behavioral transparency is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: it opens with the primary purpose, then describes accepted identifiers, then discloses the read-only behavior and host effects. Every sentence earns its place without repetition or fluff. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values are covered elsewhere. The description covers purpose, parameter identity, and behavioral side effects, which is mostly sufficient. However, it leaves workdir unexplained and does not mention the sibling bernstein_status, so an agent might struggle to choose correctly between them. This is a gap given the tool's moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50%: run_id is well described in the schema (accepts task_id or run_id), but workdir has no schema description and the tool description does not explain it either. The description merely restates run_id semantics already in the schema, adding no new meaning and leaving workdir's purpose ambiguous. With low schema coverage, the description should compensate but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool polls a verifiable handle for a run started with bernstein_run, using a specific verb and resource. It accepts either task_id or run_id, which defines its purpose well. However, it does not differentiate from the sibling tool bernstein_status, leaving some ambiguity about their distinct roles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: after calling bernstein_run, with either returned identifier. It provides some guidance on parameter inputs but does not mention bernstein_status as an alternative or specify when to choose this tool over others. The context is clear but lacks explicit exclusions or alternative comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_claimA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states that the tool changes task assignment state and makes network requests ('Host effects: makes network requests'). This goes beyond a simple 'claim' verb and informs the agent of side effects, though it does not cover authorization or reversibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences deliver the action, the state change, and the host effect without redundancy. Every sentence adds unique value, and the description is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, state-changing, network requests) and lack of annotations, the description is incomplete. It does not address parameter semantics, eligibility criteria, or usage guidance, and though an output schema exists, the missing parameter context prevents a fully capable agent from using the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, and the tool description does not explain any of the seven parameters (role, project, capability, claimer_id, max_attempts, completed_ids, claimer_card_fingerprint). The agent is left to infer semantics purely from parameter names, which is inadequate for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Claim') and resource ('next eligible task for an agent'), clearly distinguishing it from siblings like bernstein_complete or bernstein_cancel. It also clarifies the state-changing nature by stating 'This changes task assignment state on the Bernstein server.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when an agent needs to claim the next eligible task. It does not explicitly mention alternatives or when-not-to-use, but the purpose is self-evident and distinct from sibling tools. No exclusions are necessary for this action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_completeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the refusal behavior for invalid states and the host effect of making network requests. However, it does not mention reversibility, authentication, or other potential side effects beyond completing the task.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with three short sentences. It front-loads the purpose, then adds the state constraint and a host effect note. Every sentence is informative and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter mutation tool with an output schema, the description covers the core purpose and a key constraint, but parameter semantics are under-specified. An agent may need to infer the expected content of result_summary, making the description adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only mentions 'task result' generically, leaving task_id and result_summary to name inference. No additional semantic guidance is given for their format or content beyond the schema's constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Report a task result' and 'completes the task on the Bernstein server.' It also specifies the allowed worker-held states, which distinguishes it from sibling tools like bernstein_claim, bernstein_cancel, and bernstein_approve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context for when to use the tool by listing valid task statuses ('open, claimed, or in_progress') and stating that other statuses are refused. It does not name alternative tools, but the state constraint effectively communicates the appropriate usage scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_statusA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the transparency burden. It explicitly states 'Retrieves data from the Bernstein server without changing it' and 'Host effects: makes network requests,' which discloses read-only behavior and side effects. This adds useful context beyond the schema, though it does not cover details like permissions or error cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose ('Liveness, task counts, and cost in one read') and followed by parameter guidance and behavioral notes. Every sentence adds value, and there is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, return values are already covered. The description sufficiently covers the purpose, parameters, and read-only nature, making it complete for a status tool. Minor gaps exist in not fully explaining what 'liveness' entails, but overall it is well-rounded.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining both parameters: 'Pass status to include the matching tasks' and 'pass detail=true for full per-role and per-task rows.' This adds meaningful semantics beyond the bare enum and boolean in the schema, clarifying their purpose and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a read-only status endpoint for liveness, task counts, and cost. It uses specific nouns and implies a resource, but it does not explicitly differentiate from the sibling tool bernstein_run_status, so it meets 'clear' but lacks direct sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides usage guidance for parameters (pass status to filter tasks, detail=true for full rows) but does not state when to choose this tool over alternatives like bernstein_run_status. There is no explicit 'use this for server-level status' or mention of exclusions, so it is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_task_capsuleA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosure. It explicitly mentions conditional side effects (verify=true may create install audit key) and host effects ('reads files; writes files'), which is unusually transparent for a tool named 'read'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, front-loaded with the primary purpose. The side-effect disclosure is compact and informative; no filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return values need not be described. The description covers the core action, conditional mutation, and host effects, but lacks usage context around workdir and when to prefer sibling status tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema property descriptions are absent (0% coverage), so the description must compensate. It explains the verify parameter's conditional side effect, but task_id and workdir receive no semantic explanation beyond their names and schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific action ('Read a task capsule...') with a clear resource (task capsule, local journal, audit evidence). It distinguishes from siblings like bernstein_run and bernstein_status by focusing on reading capsule contents rather than executing or checking status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as bernstein_status or bernstein_run_status. The description implies a read/inspection use case but does not state exclusions or preferred alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_approveA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the tool refuses non-pending_approval statuses, completes the task on the server, and makes network requests. This gives a clear state-change and safety profile, though it does not mention reversibility or error handling specifics, which might be covered by the output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a brief 'Host effects' note. It is front-loaded with the core condition and action, contains no filler, and every clause earns its place. This is an exemplar of concise, structured writing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential behavior, the precondition, and the side effects. Since an output schema exists, return values are likely documented elsewhere. Missing details include the role of the note parameter and the relationship to bernstein_complete, but overall the description is sufficiently complete for a simple approval action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the tool description must compensate. It only vaguely refers to 'task' without explaining task_id or note. The note parameter's purpose (e.g., an approval message) is entirely unexplained, and the condition 'pending_approval' does not clarify parameter semantics enough. This is a clear gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's specific function: 'Sign off a finished result' only when the task is in pending_approval, and that it 'completes the task.' The explicit status condition ('every other status is refused') adds scope and distinguishes it from sibling tools like bernstein_complete, which may have different completion semantics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the tool: only when the task is in pending_approval. It explicitly states that any other status is refused, signaling when not to use it. However, it does not name alternative tools (e.g., bernstein_complete) or explain the exact workflow relationship, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_shutdown_orchestratorA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses a clear side effect: 'Writes the local SHUTDOWN signal file. Host effects: writes files.' It also communicates the destructive scope ('ENTIRE', 'including every run and worker'). It does not detail reversibility, permissions, or whether shutdown is graceful, but the disclosed effects go well beyond a vague mutation claim.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary action and scope. Every sentence adds value: the main behavior, the alternative tool for narrower cancellation, and the local file side effect. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose, scope, side effect, and alternative, but leaves workdir unexplained and does not describe the return value or post-shutdown state. Given the destructive nature and lack of annotations, a more complete description—especially about the parameter and consequences—would be expected. The output schema may compensate for return details, but the parameter omission remains a gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one optional parameter, workdir, with no property description (coverage 0%). The description never mentions workdir or how it affects which project is shut down. The agent is left to infer that workdir selects the project context, which is not explicitly clarified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource: 'Shut down the ENTIRE Bernstein orchestrator for this project, including every run and worker.' It clearly distinguishes itself from bernstein_cancel, which stops a single task while the orchestrator continues, making the tool's scope and purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'use bernstein_cancel to stop one task while the orchestrator keeps running.' This tells the agent exactly when to choose this tool versus the alternative, and it also implies when a full shutdown (rather than a cancel) is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_cancelA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must explicitly communicate side effects. It states 'changes task state on the Bernstein server,' 'makes network requests,' and describes behavior for terminal tasks. This fully discloses the tool's behavioral footprint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. Each sentence contributes unique information: purpose, eligible statuses, terminal-task handling, and side effects. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values need not be explained. The description covers purpose, status constraints, error behavior, and side effects. Minor gaps: no detail on invalid task_id handling or how the reason parameter influences cancellation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% parameter description coverage and the description does not compensate. It never mentions the 'reason' parameter, and task_id is only implicitly tied to 'one task.' No guidance is given on how to obtain valid task_id values beyond the schema pattern.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Cancel one task and its subtask tree.' It clearly distinguishes from sibling tools like bernstein_shutdown_orchestrator by stating 'orchestrator keeps running.' Listing the cancellable statuses further clarifies scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use: canceling a specific task tree while keeping the orchestrator alive. The enumerating of cancellable statuses and non-error on terminal tasks gives implied usage boundaries, but it does not explicitly name alternatives or exclusion scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load_skillA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries full burden. It discloses that it returns file contents as text, executes nothing, and reads files. This clearly signals a read-only, safe operation. It could further mention error behavior (e.g., not found), but the provided info is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, tightly packed with meaningful content: behavior, return type, safety, and host effect. Every word earns its place; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool structure, an output schema exists, and the description covers core behavior and safety, it is mostly complete. It could be improved by explaining how missing files are handled, but that is a minor gap. Overall, it supplies enough for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33% (only name has a description), so the description must compensate. It adds meaning by mentioning 'skill body, reference, or script file contents', mapping to the three possible loads. However, it does not explain the exact format or relationship of script/reference parameters beyond what dependencies imply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states two distinct behaviors: listing skills when name is omitted and loading skill body/reference/script contents when name is provided. The verb 'list' and 'load' are specific and the resource is well-defined, distinguishing it from the bernstein_* siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'when name is omitted' versus when a name is provided, giving clear context for both usage modes. It also notes 'executes nothing', implying it is for inspection, not execution, but it does not name alternative tools for execution, so it misses explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bernstein_runA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and delivers excellently. It discloses async (returns when queued, not finished), side effects (writes files, spawns agent processes, makes network requests), and the returned poll_after_ms. This is far beyond minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: purpose, async warning, follow-up, subtask usage, and host effects. It is well-structured and not bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex orchestration tool with side effects, the description covers all critical aspects: queuing model, duplicate-run risk, polling strategy, subtask support, and host-level consequences. Output schema exists, so return values are covered structurally.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 7 parameters and 0% schema description coverage, the description only explains parent_task_id ('pass to create the run as a subtask'). The required 'goal' and others like role, scope, priority, complexity, estimated_minutes are left unexplained, relying solely on naming and enums.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Start an orchestration run', a specific verb and resource. It clearly distinguishes from siblings like bernstein_run_status (which monitors) and bernstein_cancel by emphasizing the queuing behavior and follow-up steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use and alternatives: tells users not to re-issue while waiting (starts a second run), instructs to follow with bernstein_run_status after waiting poll_after_ms, and mentions parent_task_id for subtasks. This is exemplary usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.