Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds little beyond restating the defaults already in the schema, and leaves open behavioral questions such as what threshold makes a duration change 'significant' and how large the returned diff can get.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the verb and the enumerated diff categories, and the default behavior stated last where it lands as a useful aside. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless-by-default read-only diff tool with a fully documented schema, the description is nearly complete. The only real gap is the undefined 'significant duration changes' threshold, plus no hint of the response shape since there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters document accepted forms (run id, build number, commit prefix, 'latest'/'previous'). The description's 'defaults to previous vs latest' merely repeats what the schema already states, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete verb (diff) and resource (two runs) and enumerates the exact output categories: newly failing, fixed, still failing, added/removed tests, and duration changes. That is far more specific than a generic 'compare' but it never names a sibling such as detect_regressions, which an agent might plausibly choose instead for overlapping needs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The default ('previous vs latest') gives the agent useful context for calling it with no arguments, so usage is implied. However, there is no explicit statement of when to prefer this over detect_regressions or get_run_summary, and no exclusions are offered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds genuine behavioral information beyond that: results are grouped by the build/commit where tests first broke, and the analysis is threshold-based. It doesn't mention defaults or limits, but the schema carries those.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, both load-bearing. The qualifying criterion comes first and the grouping/output rationale second, so the agent gets the core behavior before any detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only analytical tool with no output schema and fully documented parameters, the description covers what is detected and how results are organized. It could say a bit more about what to expect in the response shape, but nothing essential to correct invocation is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and each parameter already documents its meaning and default, so the baseline of 3 applies. The description adds no parameter-specific detail such as how min_stable_passes interacts with last_n, but nothing is missing that the schema doesn't already say.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Find tests') plus the exact predicate that qualifies a regression: stable-passing and now consistently failing. This cleanly distinguishes it from sibling tools like find_flaky_tests and get_failure_clusters, which target different failure patterns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The criterion ('stable-passing and now failing consistently') plus the grouping by breaking build gives a clear context for when this is the right tool. It stops short of explicitly naming alternatives or stating when-not to use it, so it lacks the exclusion clause that would make it a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a non-read-only, closed-world, non-destructive operation, and the description adds useful context: it writes a self-contained HTML artifact to the output folder and returns its path. It would be stronger if it mentioned overwrite behavior or error handling for existing files, but it adds meaningful behavior beyond the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is a single front-loaded sentence with no wasted words. It states the action, artifact contents, destination, and return value efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the artifact, its contents, the output location, and the return value, which is sufficient even without an output schema. Minor gaps remain around overwrite behavior and failure conditions, but the core information an agent needs to invoke the tool is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description does not add semantics for last_n or filename beyond what the schema provides, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: it writes a self-contained HTML QA dashboard to the output folder and returns its path. It clearly states the artifact and its included sections, but it does not distinguish the tool from the sibling generate_quality_report or clarify how they differ.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the word 'export' and the output folder destination. There is no explicit guidance on when to use this tool instead of generate_quality_report or other reporting siblings, and no exclusion conditions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety and locality are covered. The description adds meaningful behavior beyond annotations by explaining the outage-exclusion heuristic and listing the returned artifacts: score, timeline, failure signatures, and suggested fix.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is three tight sentences. It front-loads the core purpose, then adds the important exclusion rule and return summary without repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although there is no output schema, the description names the main returned components and explains the key filtering logic. Combined with fully documented parameters and read-only annotations, this is nearly complete, with only minor gaps around output structure and scoring details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so last_n, min_flips, and min_failures are already documented with ranges and defaults. The description adds no additional parameter-level guidance, which is acceptable but does not exceed the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: detect flaky tests, defined as tests that flip between pass and fail across runs. It also narrows the concept by excluding systemic-outage failures, which separates the tool from generic failure-cluster or regression tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: call this when looking for flaky tests rather than general failures. However, it does not explicitly say when to prefer this over siblings such as get_test_history, get_failure_clusters, or detect_regressions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description usefully discloses the return artifact's format (Markdown) and section contents, but says nothing about cost, latency, or that it aggregates across runs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences that lead with format and content, then consumption targets. Every clause is informative, though the contents list is dense enough to verge on enumeration for its own sake.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must describe the return value, and it does so clearly by naming the report's sections. The only real gap is that it never relates to the last_n parameter or explains how the report scopes its data.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter last_n has 100% schema description coverage, so the schema fully documents the range constraint. The description never mentions run-scoping at all, so it adds nothing beyond the schema; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

A specific verb (generate) and resource (QA health report) with an enumerated list of the report's contents, so an agent knows exactly what it produces. It doesn't distinguish itself from the sibling export_dashboard, which plausibly overlaps as an aggregate-output tool, so it stops short of 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Ready to paste into Slack, Confluence or a Jira comment' implies the intended consumption context, but there is no explicit when-to-use-this vs. the many per-metric siblings (get_pass_rate_trend, find_flaky_tests, detect_regressions) that this report aggregates. Usage is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is a read-only, closed-world operation. The description adds meaningful behavioral detail beyond annotations: how failures are normalized and which root-cause classes are assigned, though it does not describe the exact return structure or limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the tool's core action and followed by a direct RCA usage note. Every sentence carries useful information with no repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the read-only annotations and fully documented parameters, the description is largely complete for an analytical grouping tool. It explains the clustering method and classification categories, though with no output schema it could have said more about the returned cluster structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for both parameters, including accepted values for `run` and the default behavior. The description adds no parameter-specific syntax or semantic detail beyond what the schema already provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb, "Group failures," and clearly defines the resource and method: normalized root-cause signatures with four classification categories. It is distinguishable from generic summary or regression tools, though it does not explicitly name sibling tools to differentiate itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context by calling itself "the starting point for an RCA," which tells an agent when this tool is appropriate. It does not include explicit exclusions or name alternative sibling tools, so it falls short of the top score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, openWorldHint=false), so the description only needs to add context. It usefully discloses that data is aggregated per run and can be narrowed to a suite, but says nothing about the sampled run window vs. all runs or the shape of the series returned. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence with the metric and scope front-loaded and zero filler. It is efficient, though the terseness leaves room for one more clause about the trend window.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read-only query with no output schema, the description covers what is measured and the filter option. It is adequate but leaves return shape (a series of run/pass-rate points) and default behavior implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented, and the description's example 'ui.checkout' mildly clarifies the expected dotted suite-name format. The last_n window parameter is not addressed at all in the description, so it adds only marginal value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific metric (pass rate), its aggregation granularity (per run), its ordering (over time) and its optional scope (overall or one suite). An agent can distinguish it from aggregate siblings like get_run_summary or detect_regressions, though no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no reference to alternatives such as get_test_history (per-test trend) or compare_runs. The agent must infer from the 'over time' phrasing that this is the trend-over-runs view.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Read-only and closed-world behavior is already covered by annotations. The description adds that results include every failure and skipped test, which hints at a potentially large payload, but it does not discuss size, pagination, or formatting.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence lists exactly what the tool returns. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter schema and absence of an output schema, the description sufficiently enumerates returned data. It could say more about payload size or structure, but it covers the essential contents for this read-only summary tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single optional 'run' parameter has full schema coverage, including default behavior and accepted identifier formats. The description mentions only 'one run' and adds no parameter semantics beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource (one run) and enumerates the returned data: totals, pass rate, per-suite breakdown, every failure and skipped test. This clearly distinguishes it from trend, comparison, and flaky-test siblings, though it does not explicitly name those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for one run' implies the scope, but there is no explicit guidance on when to choose this over get_pass_rate_trend, compare_runs, get_failure_clusters, or list_runs. An agent must infer usage from the content list alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds useful behavior (trend across the window, degradation flags) but leaves open what 'flags degradation' means and whether the trend requires multiple runs in the window.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense sentence with no filler, and the resource plus scope are front-loaded. Slightly fragmented phrasing ('across the window') but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-optional-parameter read tool with annotations and no output schema, the definition is broadly sufficient to call it. It does not clarify ordering of results, what constitutes a degradation flag, or the minimum data needed for a trend, which are the remaining gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both top_n and last_n fully documented including defaults and bounds, so the schema carries parameters. The description only weakly reinforces them via 'across the window', adding little beyond the schema — baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a clear resource (slowest tests) with scope modifiers — latest run plus the duration trend across the window — and adds degradation flagging. It is distinguishable from siblings such as get_pass_rate_trend and find_flaky_tests by resource, though it doesn't name an alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is for performance analysis. There is no explicit when-to-use or when-not, and no disambiguation from detect_regressions, which also deals with performance problems — a real ambiguity given both mention degradation/regression behavior.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint=true, openWorldHint=false), so the description's value lies in disclosing the return shape — status, duration, message per run — which matters since there is no output schema. It does not state ordering, pagination, or what happens when the substring matches multiple tests.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the return fields come first and the matching rule plus examples follow. Nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read tool with no output schema, the description covers purpose, filter semantics, and return fields — enough to call it correctly. Gaps are minor: run ordering, multi-match behavior, and empty-result behavior are unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline would be 3; the description earns a bump by clarifying that matching is substring-based ('id contains the query') and supplying concrete id examples like 'apply_coupon' and 'ui.checkout'. It still says nothing about last_n, which the schema handles alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource (run-by-run history of a test) and enumerates what each row contains (status, duration, message), plus the filter semantics ('tests whose id contains the query'). It is clearly distinct from get_run_summary and list_runs by being per-test across runs, though it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the tool for 'show me this test's recent results', but the description gives no explicit when-to-use, when-not, or routing to get_run_summary / find_flaky_tests for related questions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety and scope are covered. The description adds the ordering guarantee ('oldest first') and the returned field set, which is real value beyond annotations. However, it says nothing about volume — that the default returns every run, and that the schema caps last_n at 500 — which matters for an unbounded listing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the verb and resource, with the returned fields and ordering as compact trailing qualifiers. Nothing is wasted and nothing important is buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully enumerates the returned fields, and the single optional parameter is fully covered by the schema. What is missing is a note on result volume/pagination behavior given the 'all runs' default, which is the one thing an agent could get wrong when calling this at scale.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single optional parameter last_n is fully documented in the schema, including its 2–500 bounds and default of all runs. The description adds nothing about last_n, so baseline 3 applies. There is a minor friction: the tool is described as 'oldest first' while last_n is framed as 'most recent N', which the description does not reconcile.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List available test runs') and adds scope detail by naming the fields returned (build, commit, date, headline pass rate) and the ordering ('oldest first'). It implicitly distinguishes itself from the analytical siblings (get_run_summary, compare_runs, detect_regressions), which operate on runs rather than enumerate them, but never states that differentiation explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the discovery/entry-point tool that produces the run identifiers the other siblings consume. There is no explicit 'use this when...' clause, no mention of when to prefer it over get_run_summary, and no note about the cost of listing all runs before narrowing with last_n.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.