Server Quality Checklist

Tool Scores

• run_engagementA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description extensively discloses behavior beyond the readOnlyHint annotation: determinism ('same seed always replays the identical engagement'), STALEMATE outcomes and the packages_expended reason, tactical-mode output differences, and the crucial flag that the data is 'Notional data.' No contradiction with readOnlyHint exists — the deterministic run-and-return semantics are consistent with a read-only computation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph with the core action front-loaded ('Run a single deterministic force-on-force engagement to completion') followed by the return-record enumeration. It runs long due to the output list, but every clause earns its place given there is no output schema to carry that information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema present, the description's detailed enumeration of the return record (winner, reason, duration, phase timeline, CEP breakdown, LOB/intercept counts, timestamps, end states, event log, and tactical-mode additions) serves as the de facto output documentation. Combined with the rich input schema (featured seeds, mode semantics, config defaults), an agent has everything needed to invoke and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents mode, seed, and every nested config_overrides field with units, defaults, and ranges. The description adds only marginal parameter context, e.g., that tactical mode adds objective and sortie-tally outputs. Baseline 3 is appropriate when the schema carries the parameter documentation burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Run a single deterministic force-on-force engagement to completion' and enumerates the full return record. The 'single' qualifier distinguishes it from the sibling sweep_seeds, and the output enumeration (winner/STALEMATE, phase timeline, CEP breakdown, event log) leaves no ambiguity about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on what the tool does and how orbit/tactical modes alter behavior, which informs when an agent would call it. However, it never explicitly names alternatives or exclusion conditions — it doesn't say to use sweep_seeds for multi-seed exploration or compare_configs for configuration comparisons. Differentiation from sweep_seeds is only implicit in the word 'single'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sweep_seedsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only declare readOnlyHint=true, so the description carries behavioral disclosure. It adds valuable details: aggregation is computed server-side, per-run event logs are not returned, and max count is 1000. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences plus a short max-count clause, with the operation and scope front-loaded. No filler; every sentence contributes to selection or invocation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a batch-analysis tool with no output schema, the description enumerates the return aggregates in detail and clarifies the no-logs constraint. It is sufficient for an agent to decide and call correctly, though it doesn't describe error handling or exact output structure, which is acceptable given the schema richness elsewhere.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds the seed-range formula and restates max count, but parameter-level meaning is already well documented in the schema, including mode explanations and featured seed examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Run'), resource ('count consecutive seeds'), scope ('under one configuration'), and output ('aggregate statistics only'). It enumerates the exact statistics returned and explicitly notes that per-run event logs are not returned, which differentiates it from run_engagement without needing to open that tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: run a batch of seeds and get only aggregate statistics. It routes to run_engagement for notable seeds and states that per-run logs are absent, implying when to switch. It doesn't explicitly cover compare_configs, describe_model, or get_config_schema, but the primary use case is well defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_configsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  readOnlyHint=true already covers the safety profile, and the description adds useful behavioral context: the paired-seed rationale, the artifact list returned, and the count cap. It does not contradict the annotation, and the added detail goes beyond what readOnlyHint alone provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all high-signal: purpose and output list in the first, a concrete use case in the second, and the operational limit in the third. There is no filler or redundant elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does the necessary work of telling the caller what will be returned. It also explains the experimental design, gives a representative use case, and caps the seed count. Combined with the highly detailed input schema, an agent has enough context to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameter semantics are already documented in the input schema. The description's references to same consecutive seeds and max count of 500 largely restate schema constraints rather than adding new parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Run two CONFIG variants over the SAME consecutive seed range') and clearly identifies the tool's resource and experimental design. It also enumerates the concrete outputs (aggregate statistics, paired deltas, flipped seeds, plain-language summary), making it easy to distinguish from single-run or unpaired sweep tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool for doctrine questions and explains why the paired same-seed design is appropriate ('terrain and emplacement luck cancel out'). It does not explicitly name alternatives such as run_engagement or sweep_seeds or state when not to use this tool, so it falls just short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_modelA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, but the description adds value by detailing the content of the output (assumptions and limitations). No contradictions. The behavioral context is well explained beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that are front-loaded and contain all necessary information without redundancy. Every sentence serves a purpose, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description comprehensively lists the return content and mentions the tool's value in understanding model limitations. It is complete for a descriptive read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist (0 parameters, schema coverage 100%). The description is not required to add parameter semantics, and it does not. Baseline score of 4 is appropriate for this case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states that the tool returns modeling assumptions and lists specific components (DF measurement, bearing-error model, etc.). It clearly differentiates from sibling tools like 'run_engagement' which are for running simulations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises to read the output before drawing conclusions, providing clear when-to-use guidance. It also implies this tool is for understanding model limitations, not for running simulations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_config_schemaA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds that it's generated from the validation table ensuring no drift, providing complete behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each carrying essential information without waste; first sentence details returns, second adds non-drift property.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool without output schema, the description thoroughly covers what is returned and a key behavioral property, making it complete for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters; description adds meaning by enumerating output contents (path, unit, default, etc.) and mentioning non-overridable parameters, which is valuable beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies 'Return every parameter accepted in config_overrides' with details like path, unit, etc., and distinguishes from sibling tools by focusing on schema retrieval rather than execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly describes what the tool returns and mentions the non-overridable parameters, guiding when to use for understanding tunable parameters, but lacks explicit when-not-to-use or comparison with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.