Server Quality Checklist

Tool Scores

• generate_jmxA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses the return value (written path and inline jmx) and the truncation behavior at 200,000 characters. However, it does not mention side effects like file overwriting, needed permissions, or failure behavior, leaving some important behavioral traits unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: purpose, usage, then return behavior. Every sentence contributes meaningful information, and there is no redundant text. It fits the tool's complexity without unnecessary prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters (3 required) and a nested workload object, the description is too sparse. It omits entirely the required 'correlation_report' and the semantics of the workload parameters. Although an output schema exists, the input-side guidance is incomplete, leaving the agent to guess how to construct valid arguments.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the input schema. It only clarifies that 'flow' can be an object or a persisted flow path, but it does not explain the required 'correlation_report' or the complex 'workload' object with its aliases and defaults. The 'output_path' parameter is also unmentioned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and artifact: 'Build an Apache JMeter 5.6.3 plan from a correlated flow and workload.' It clearly distinguishes this from sibling tools like edit_jmx and run_test by indicating it is the generation step in a pipeline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use after correlate_flow', providing a clear workflow position. It also notes that the tool accepts either a flow object or a persisted flow path, which helps the agent choose the right input format. It does not explicitly name alternatives or exclusions, but the sequencing guidance is valuable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_resultsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. It mentions it 'Returns summary metrics and findings,' giving some insight into output behavior. However, it does not state whether the operation is read-only, what side effects it may have, or any error conditions. This is minimal transparency but not misleading, so a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a distinct purpose: stating the core function, providing usage timing, and describing the return value. It is front-loaded with the main action and contains no extraneous information. Conciseness is excellent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter) and has an output schema, but the description leaves gaps. It doesn't explicitly define results_path or address potential side effects or prerequisites beyond timing. While concise and clear about the core function, it misses key details for a fully self-contained description, so a 3 is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (results_path) with zero description coverage. The description does not explicitly explain what results_path should contain, though it is implied to be the path to the JTL file. With 0% schema coverage, the description fails to adequately compensate for the missing parameter documentation, resulting in a score of 2.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Summarise a JTL and rank bottleneck findings at p95/p99.' It specifies the resource (JTL) and the precise action (summarize/rank), which differentiates it from siblings like run_test or compile_report. The purpose is unambiguous and action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a timing condition: 'Use after a completed run.' This is clear usage context. However, it doesn't mention when not to use the tool or explicitly name alternatives, though the sibling tool list provides implicit differentiation. This is clear context without exclusions, warranting a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• correlate_flowA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It explains the core operation (detect and rewrite), the accepted input forms (flow object or flow_path), and the outputs (correlation report, rewritten flow, new flow_path). However, it does not disclose whether the original flow is mutated in place, whether persistence overwrites existing flow paths, or any environment prerequisites beyond 'after import_traffic.' These gaps justify a mid-range score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no fluff. The first sentence is a precise one-liner stating the purpose, the second gives a usage directive, and the third lists the return values. Every sentence contributes meaning, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has moderate complexity with an output schema, so return values are covered elsewhere. The description covers the core parameter and usage timing, but it omits the second parameter's purpose and side-effect details. Given that annotations are absent and the description is the only source of behavioral context, it is not fully complete, though it covers the essentials. This warrants a 3.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 2 parameters with 0% description coverage. The description compensates partially by explaining the `flow` parameter, noting it accepts either an object or a persisted flow_path. It does not explain `confidence_floor`, leaving that parameter semantically opaque. Because the most important parameter is clarified but the second is ignored, a score of 3 reflects the partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific function: 'Detect dynamic values and rewrite the flow with JMeter variables.' It names the resource (flow) and the specific action, distinguishing it from sibling tools like analyze_results and run_test. The mention of 'Use after import_traffic' further situates its unique role in the workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit usage cue: 'Use after import_traffic.' This gives clear contextual guidance on when this tool should be invoked. It does not mention when not to use it or alternative tools, so it stops short of a full 5, but the clear sequencing earns a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• correlate_with_signalpilotA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does well by revealing the return type (SignalPilotReport) and the failure mode when the CLI is missing (available=False). It does not mention side effects, but for a merge/correlate operation this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences with no fluff. The primary action and use case are front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main use case and a key edge case (missing CLI), and the output schema exists. However, missing parameter explanations and the absence of a broader behavioral context leave gaps for a 3-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no parameter-level details. 'Run window' only weakly maps to run_id; namespace and context remain completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action (merge Kubernetes RCA from SignalPilot) and a clear scope (recorded run window). The 'use when' phrase further distinguishes it from sibling tools by tying it to server-side latency explanation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: when client-side latency needs server-side explanation. It does not mention alternatives or exclusions, but the condition is specific enough to guide selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ensure_environmentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full transparency burden. It discloses that the tool 'heals' and returns actions taken, implying side effects like downloads or configuration changes. However, it doesn't explicitly state what specific actions are performed, whether permissions are required, or the potential impact on the system. The presence of the 'allow_download' parameter hints at downloads, but the description itself doesn't elaborate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, two sentences plus a param guidance. The first sentence front-loads the core purpose, followed by usage context and return information. Every sentence provides useful information without redundancy, making it well-structured and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, when to use it, return value, and one key parameter. An output schema exists, so return value details are optional. However, it could be more complete by mentioning prerequisites (e.g., network access for downloads) or what a 'heal' might entail, but these gaps are minor given the output schema and clear usage guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It explains force_docker well: 'Set force_docker=true to prefer Docker when docker_available (even if native is ready)'. However, it does not explain allow_download beyond its name and default, leaving its semantics partially inferred. This partial coverage justifies a 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'Diagnose and heal the local JMeter runtime (JDK, JMeter, plugins, Docker)'. The verb 'diagnose and heal' is specific, and the resource scope is explicit. This distinguishes it from sibling tools like diagnose_environment, which likely only diagnoses but does not heal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to call: 'Call when the host may lack a usable runtime.' It also gives usage direction for the force_docker parameter. However, it does not explicitly mention alternatives or when not to use this tool (e.g., if only diagnosis is needed), so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_jmxA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses critical behavioral traits: 'Writes a new plan by default; set overwrite=true only to replace plan_path' and lists return values ('Returns path, summary, ops_applied, and next_action'). This is adequate transparency for a mutation tool, though it stops short of covering error cases or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and thoroughly front-loaded. The opening sentence states the core purpose, followed by usage context, operation examples, default behavior, and return values. Every sentence adds information without redundancy, making it an exemplary model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, lack of annotations, and presence of an output schema, the description covers the essential aspects: when to use, what to provide, and what to expect in return. It could enrich the details of individual operations, but the provided information is sufficient for an agent to decide and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It explains the overwrite parameter's purpose and gives examples of operations (e.g., set_workload with fixed/ramp/stress/burst). However, plan_path and output_path are only superficially referenced, and the nested operation parameters are not detailed beyond the op enum list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Apply structured edits to an existing JMX plan and write a new file.' It uses a specific verb and resource, and lists example operations, making its purpose unambiguous. It also distinguishes itself from the sibling generate_jmx by explicitly positioning itself as the follow-up editing step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use after generate_jmx (or with a hand-written single-ThreadGroup plan).' This tells the agent when to use the tool and what inputs are appropriate. It doesn't explicitly list alternatives or exclusions, but the given context is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• import_trafficA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses type detection behavior (kind=auto), return format (flow JSON plus request_count), and that response details are omitted by default unless include_response_detail=true. This is valuable beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four tightly worded sentences. Every sentence adds distinct information—purpose, workflow placement, auto-detection, and response detail behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and no annotations, the description provides key context: when to use, what it returns, and a critical default behavior. It omits details on base_url and include_static, but the output schema and sibling tools fill some gaps. Overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It explains kind and include_response_detail but leaves base_url and include_static ambiguous, though source_path is implied by the import context. Partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool imports HAR, OpenAPI, or Postman traffic into a normalised Flow. This specific verb+resource combination distinguishes it from sibling tools like run_test or analyze_results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use at the start of scripting,' placing this tool early in the workflow. It doesn't name alternative tools or exclusions, but the context signals provide clear separation from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_sloA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior itself. It reveals the return type 'SLOVerdict' and the key field 'compliance_pct', giving some insight into expected output. Yet it never explicitly says whether the operation is read-only or has side effects, and there is no discussion of error handling or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no superfluous content. The first sentence is a dense statement of purpose, and the second adds usage and return value, making every word valuable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (two string parameters) and the presence of an output schema, the description covers the essential context: purpose, usage, and output. It mentions the return object and its key metric, which is adequate for a CI gate without requiring extensive warnings or side-effect notes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It effectively maps 'results_path' to 'JTL results' and 'slo_path' to 'slo.properties', clarifying the purpose of each parameter beyond the terse schema titles. It does not detail file formats but that is likely sufficient for a straightforward file-path tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Gate JTL results against an slo.properties file', which is a specific verb (Gate) plus resource (JTL results and slo.properties). This clearly differentiates it from sibling tools like analyze_results, and the second sentence confirms its role in CI pass/fail decisions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use for CI pass/fail decisions', providing clear context on when to invoke the tool. However, it does not mention any alternatives or when not to use it, so it falls short of the full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_workloadA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and excels. It discloses the dependency on ensure_environment, the iterative process (reuses same plan, overrides thread count/duration), the return value ('DiscoveryOutcome plus provisioning and a recorded run_id'), and parameter constraints like run_id regex and overwrite behavior. This is rich behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences pack a wealth of information without waste. The first sentence states the core purpose, the second covers workflow and process, and the third covers return values and constraints. Excellent structure and front-loading.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential workflow (dependency, step mechanics, return type) and key constraints (run_id format, overwrite). It omits explanation of the discovery and guardrails parameters, but guardrails schema provides its own descriptions. Given the output schema exists, the description is sufficiently complete for most use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics for run_id (validation regex, collision/overwrite) and implicitly for plan_path (via 'same plan'), but does not explain the discovery object or guardrails. Guardrails has internal descriptions, but for the top-level parameters, coverage is incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action: 'Find a capacity knee by stepping concurrency via -Jperfsage.threads.' This clearly distinguishes the tool from siblings like run_test or analyze_results by its discovery-focused purpose. The verb and resource are explicit and non-tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (find a capacity knee) and notes a prerequisite ('Calls ensure_environment first'). However, it does not explicitly mention alternatives or when-not-to-use compared to sibling tools, so it stops short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_testA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and meets it by disclosing always-on guardrails, the ensure_environment call, return fields, run_id validation rules, overwrite semantics, and force_docker preference. This is substantial behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: purpose, prerequisite, return value, guardrail requirement, run_id validation, and force_docker. It is front-loaded with the core purpose and avoids fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description provides a solid high-level picture: what it does, prerequisites, return value, and key constraints. It omits details about some parameters and the exact behavior of guardrails, but the parameter names and defaults plus the output schema cover some of that. It is adequate for an agent to select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for run_id (regex and collision handling), acknowledged_risk alias, and force_docker, but does not explain plan_path, properties, timeout_seconds, generate_html_report, or overwrite beyond the collision note. With eight parameters, this is only partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Execute a JMX plan' with a specific verb and resource, and further specifies guardrails and streamable JTL output. It distinguishes itself from siblings by noting it calls ensure_environment first and is the execution step in the workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is the tool for running JMX plans and even notes the prerequisite of ensure_environment. It does not explicitly name alternative tools or when not to use it, but the context and sibling list make the execution role obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diagnose_environmentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It clearly labels the tool 'Read-only' and states the return type and fields ('EnvironmentReport with ready/java/jmeter/plugins fields'), giving agents confidence that this is a safe inspection. It does not cover potential error conditions, but for a zero-parameter read-only tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a front-loaded action phrase and a clean usage statement. Every sentence adds value: one identifies scope and safety, the other gives timing and return shape. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-argument diagnostic tool with an output schema, the description provides the essential purpose, read-only safety, usage timing, and return structure. It is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters and schema coverage is 100%, so the baseline for parameter semantics is 4. The description needs to add no parameter-specific details, and it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Read-only environment diagnosis for Java, JMeter, plugins, and Docker,' which provides a specific verb and resource scope. It clearly distinguishes the tool from siblings like ensure_environment by emphasizing a non-mutating readiness check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Use before planning work when you only need readiness details' gives explicit contextual timing and conditions for use. It does not explicitly name alternatives or exclusions, but the phrase 'when you only need readiness details' implies other tools handle additional needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compile_reportA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for disclosing side effects and behaviors. It goes beyond a simple 'compile' statement by explicitly mentioning optional evaluation of SLOs, SignalPilot, and Reveal, plus the knee-results edge-case behavior. It also states what is returned ('verdict, written paths, and the report dict'), implying file-writing side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each delivering distinct value: core purpose, sequencing, special-case guidance, and return summary. No filler or redundant phrasing; it is front-loaded with the most essential information and remains compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 params, output schema present), the description covers typical usage, a special discovery-run scenario, and return value. It does not elaborate on parameter relationships (e.g., namespace), but the existing schema and output schema reduce the need for that detail. Overall, it provides sufficient context for an agent to select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explicitly explains the non-obvious use_knee_results parameter and hints at slo_path and use_reveal through 'Optionally evaluates SLOs ... and Reveal.' However, namespace and the exact relationship between slo_path and SLO evaluation are left implicit. It adds meaning but does not fully define every parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Compile Markdown/HTML/JSON artifacts with a verdict-led summary,' which clearly specifies the action (compile), the resources (artifacts), and the purpose. It distinguishes itself from sibling tools like analyze_results and evaluate_slo by focusing on report generation rather than analysis or evaluation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Use after analyze/evaluate' explicitly indicates the correct sequencing and positions the tool as a downstream step. The additional instruction for discovery runs—'set use_knee_results=true to analyze/SLO the knee step JTL when a knee exists'—provides scenario-specific guidance that helps the agent decide when to adjust the default behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.