Server Quality Checklist

Tool Scores

• omp_continueA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide destructiveHint=true and openWorldHint=true, so the agent knows this tool mutates state. The description adds that it is 'bounded to remaining attempts within max_attempts', which is useful context about limits. However, it does not explain what gets destroyed (e.g., previous feedback? session progress?), nor does it clarify side effects or authorization needs. This adds some value but is not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences. The first sentence front-loads the primary verb and resource, and the second adds a critical constraint. No redundant or irrelevant information is present. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that annotations and an output schema exist, the description covers the essential purpose and a key constraint. However, it does not mention prerequisites (e.g., the session must exist and be in a state that allows continuation) or what happens if remaining attempts are exceeded. For a tool that modifies state (destructiveHint=true), a note on reversibility or confirmation would improve completeness, but the description is nearly adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33% (only the `feedback` parameter has a description in schema). The tool description clarifies that `job_id` identifies the session to resume, which adds meaning beyond the schema (which only has minLength). However, `timeout_minutes` is not mentioned in either the schema or the description, leaving its purpose implicit. The description partially compensates for low coverage but leaves a gap for one parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool resumes an OMP session with targeted feedback to correct defects, avoiding starting fresh. The verb 'Resume' and resource 'same OMP session' are specific, and the phrase 'bounded to remaining attempts within max_attempts' adds scope. This differentiates from siblings like omp_run_compact (which starts new tasks) and omp_cancel (which ends sessions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the use case (correcting acceptance defects without starting fresh) and mentions a bounded constraint (remaining attempts). However, it does not explicitly state when NOT to use this tool or name alternatives (e.g., 'Use omp_run_compact for new tasks' or 'If the session has no remaining attempts, use omp_delegate instead'). The usage is implied but lacks direct exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_run_batch_compactA
  4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond what annotations provide: it reveals the bounded rolling pool (max_parallel 1-10), dependency DAG enforcement, write-ownership safety, wait timeout with completion or partial result handling, and return of group_id for follow-up. Annotations only indicate destructive and open-world hints, so the description significantly enhances transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the core action, and contains no redundant information. Every sentence adds value, describing the submission, execution constraints, and timeout behavior. It is optimally concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, DAG, safety, timeout) and the presence of an output schema, the description covers the main behavior thoroughly. It explains the batch submission, concurrent execution, dependency enforcement, and timeout handling. Minor gaps exist (e.g., error handling for failed tasks), but the output schema likely addresses return values, making the description largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 8 parameters have descriptions in the input schema (100% coverage), so the baseline is 3. The description mentions max_parallel and wait_seconds ranges but mainly provides behavioral context rather than new parameter-level meaning. It does not add insight beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool submits a decomposed group of OMP tasks with dependency DAG and write-ownership safety, and returns compact aggregated results. The verb 'submits' and resource 'batch of OMP tasks' are specific. However, it does not explicitly differentiate from sibling tools like omp_run_compact, relying on the title's 'Preferred for Multi-task' which is not part of the description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for multi-task batches by mentioning 'decomposed group' and a 'detached group runner', and suggests follow-up with omp_wait_group if incomplete. However, it lacks explicit guidance on when to use this tool versus alternatives (e.g., omp_run_compact for single tasks) or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_run_compactA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explains the creation, waiting, and return behavior (compact summary vs. job_id/status running) beyond what annotations provide. Annotations indicate destructiveHint=true (mutation), and the description corroborates and elaborates on the lifecycle. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, front-loads purpose, and every sentence adds value: preferred entrypoint, process, success return, timeout case. No extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core behavior and return values, which is sufficient given the output schema. However, it omits important behavioral details like retry logic (max_attempts), overall timeout (timeout_minutes), and the role of acceptance/supervisor_brief. For a tool with 7 parameters, a more complete description would address these.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 57% schema coverage, the schema already describes four parameters. The description only mentions wait_seconds default (60s) and does not explain goal, cwd, acceptance, max_attempts, timeout_minutes, or supervisor_brief. It misses the opportunity to add meaning beyond the schema, particularly for the less-documented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is the 'preferred entrypoint for single substantive execution tasks,' specifies the action (creates a delegated OMP task, launches runner, waits), and differentiates itself from siblings like omp_run_batch_compact by focusing on single tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description positions this tool as the preferred choice for single tasks and references subsequent polling with omp_wait for running jobs. It does not explicitly list when not to use it or contrast with sibling tools like omp_delegate or omp_run_batch_compact, but the context is clear enough for an agent to infer appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_waitA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, idempotent, non-destructive. Description adds value: polling behavior, timeout defaults and max, and that it returns status only at terminality or deadline. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: two sentences that cover purpose, behavior, parameters, and usage guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a polling wait with only two parameters, the description is largely complete. The output schema exists, so return value is covered. Minor gap: doesn't specify that the job must already exist (vs. being started elsewhere).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description explains both parameters: job_id (required, implicit), wait_seconds (with default 30s, max 60s). It adds meaning beyond schema types and constraints, but the parameter names are self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes a specific action: waiting for an OMP job to reach terminal status. It clearly distinguishes itself from siblings by mentioning polling with configurable timeout and return of status/summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the use case: waiting for an existing job to completion. The description includes a behavioral guideline ('Avoid polling in a tight loop'), but does not explicitly contrast with siblings like omp_wait_group or omp_cancel.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_cancel_groupA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds valuable context: 'Writes a cancellation request that the detached group coordinator handles safely' – this explains that cancellation is asynchronous and not immediate, which is critical behavioral detail not in annotations. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all essential: action, mechanism, usage context. No wasted words. Could be slightly more concise by merging first two sentences, but overall excellent structure and front-loading.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (2 parameters, output schema present), the description covers the core behavior and usage context well. It does not explain the output schema content or return values, but since an output schema exists, the description does not need to. The lack of details on edge cases (e.g., if group already done) is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both parameters have descriptions). The description does not elaborate on parameter details beyond what the schema already provides. With full coverage, baseline 3 is appropriate as the description adds no extra parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'immediate cancellation of a running batch task group and its child tasks'. It uses specific verbs ('cancel', 'stop') and resources ('batch task group', 'child tasks'), and distinguishes from siblings by referring to a 'group' context, which the sibling list includes 'omp_cancel' (likely single-task cancellation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Call this immediately when the user asks to stop a batch.' This sets clear when-to-use context. However, it does not mention when NOT to use it (e.g., if the group is already finished) or alternatives among siblings like 'omp_cancel' for single tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_delegateA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this tool as destructive (destructiveHint: true) and not read-only (readOnlyHint: false), and the description reinforces that by saying ownership is transferred. The description adds value by disclosing that it is a low-level delegation entrypoint and that it returns immediately, which helps the agent understand the async behavior. However, it does not elaborate on what destruction entails or if there are side effects beyond ownership transfer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the key verb and immediate behavior in the first sentence, and the alternative suggestion in the second. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's relative complexity (6 parameters, async delegation, destructive, with an output schema for return value), the description covers the high-level intent but omits details about parameter semantics and what exactly is returned beyond 'job_id'. The output schema exists but is not referenced, and return behavior is only partially described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%, so the description should compensate but does not explicitly describe each parameter. The description only mentions 'goal' and 'cwd' contextually, while schema provides descriptions for most parameters. The description does not add meaning beyond the schema for parameters like 'acceptance', 'max_attempts', 'timeout_minutes', or 'supervisor_brief'. Baseline is acceptable but not exemplary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's action ('delegate' an execution task to OMP), the resource ('execution task'), and the return value ('returns immediately with job_id'). It also clearly distinguishes itself from the sibling tool 'omp_run_compact' by calling it a low-level entrypoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides direct guidance on when to use this tool and when to prefer an alternative. It says 'Prefer omp_run_compact for standard workflows to avoid separate delegate/wait round trips', explicitly naming the sibling and the trade-off. This is the highest level of usage guidance an agent could want.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_resultA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the description does not need to re-state safety. It adds value by disclosing that the tool is for 'terminal' jobs and retrieves 'attempt logs,' which are behavioral details beyond the annotations. Score of 4 reflects strong transparency with minor room for additional context (e.g., what happens if the job is not terminal).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the purpose, and every sentence earns its place. The first sentence states the function, the second defines when to use it. No waste, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter and an output schema exists, the description does not need to explain return values. It covers the key context: what the tool retrieves, when to call it, and when not to. A slight gap is not specifying that the job must be terminal, but this is implied by 'terminal OMP job.' Score of 4 reflects strong completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the tool description must compensate. The description does not add meaning to the 'job_id' parameter beyond what the schema provides (required string with minLength). However, the parameter is simple and self-explanatory. Baseline 3 is appropriate as the description does not actively add value but the parameter is trivial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full details of a terminal OMP job, specifically 'finalResponse and attempt logs.' The verb 'retrieve' is specific, and the resource 'terminal OMP job' is well-defined. This distinguishes it from siblings like 'omp_run_compact' or 'omp_cancel' which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'call this only when minimal acceptance check fails, high-risk operations occurred, or the user explicitly asks for full inspection; do not call this after every successful compact run.' This clearly tells the agent when and when not to use this tool, differentiating it from routine monitoring tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_wait_groupA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare this as read-only (readOnlyHint: true), idempotent, and non-destructive. The description adds critical behavioral details beyond annotations: it explains two possible outcomes (full results on completion, minimal progress counts on timeout), clarifies the poll-up-to pattern, and confirms no task summaries leak on timeout. This goes beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, both front-loaded with the core action and outcome. No wasted words; every sentence adds value about behavior or return conditions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists (not shown but noted in signals), the description does not need to document return fields. It covers the key behavioral scenarios (success and timeout), parameter constraints, and source of group_id. Slight deduction for not mentioning whether the call can be re-invoked after timeout to continue waiting, but annotations imply idempotency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaning beyond the schema: it explains that wait_seconds controls polling duration with a clear range and default, and that group_id must come from omp_run_batch_compact. The description does not describe format or validation beyond schema, but it effectively contextualizes both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Wait for'), identifies the resource ('OMP task group'), and clearly defines the scope (terminal status, polling behavior). Distinguished from siblings like omp_wait by specifying 'Batch Task Group' and referencing omp_run_batch_compact.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (after omp_run_batch_compact) by requiring group_id, but does not explicitly state when not to use it or name alternative tools for other cases. The context of sibling tools provides partial guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• omp_cancelA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and idempotentHint=true. The description adds valuable behavioral context: the cancellation is immediate, the 'detached runner owns the OMP child PID' and terminates the process tree safely. No contradiction with annotations. It could mention whether a cancelled job can be restarted, but otherwise rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three crisp sentences with no wasted words. The key action verb is first, followed by the mechanism, then the usage imperative. Every sentence earns its place, and the critical usage guidance is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 2 parameters, an output schema, and sibling tools. The description clearly handles the core purpose and usage constraints. It lacks a note on the output schema (e.g., whether it returns success/failure), but with an output schema present, the description need not explain return values. It could mention what happens if the job_id doesn't exist or is already cancelled, but overall sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not provide detailed parameter semantics but it adds that the operation is per 'one exact' job (implying the job_id parameter uniquely identifies it). The reason parameter is not described, but given the tool's focused purpose and the default value in the schema, the agent can infer it. A brief mention of reason's purpose would push to 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('cancel') and resource ('delegated OMP job'), clearly distinguishing the tool from siblings like omp_cancel_group. It states exactly what the tool does (request immediate cancellation of one exact job) and how it works (the runner terminates the process tree safely).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance ('Call this immediately when the user asks to stop') and what not to do ('do not inspect PIDs or manually kill processes first'). This effectively differentiates from alternative approaches and sets clear guardrails for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.