Analyze Test Failures

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description is solely responsible for behavioral transparency. It only states that it analyzes and groups, without disclosing output format, prerequisites, side effects, or error behavior. The term 'approved' suggests a constraint but is not explained, leaving a meaningful transparency gap for a tool that produces a categorized analysis.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that packs the core action, input, and outcome without any filler. Every word adds value, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema is present, so the description should explain the return value. It hints at the result ('group them by probable failure category') but does not specify the structure of the output, potential edge cases (e.g., no failures), or how 'approved' is determined. For a tool with one parameter and no nested objects, this is a moderate completeness gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers the single parameter fully with a clear description ('Path to the report relative to the approved reports directory, such as json/playwright-results.json.'), earning a baseline of 3. The description's mention of 'approved Playwright JSON report' aligns with the schema but adds no extra semantic detail beyond what is already provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Analyze'), the resource ('failed, timed-out, and flaky tests from an approved Playwright JSON report'), and the outcome ('group them by probable failure category'). This sharply distinguishes it from sibling tools like list_test_runs or generate_bug_report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you have an approved Playwright report with failures that need categorizing, but it avoids explicit alternatives or exclusions. It doesn't mention when to choose this over generate_bug_report or get_test_run_summary, leaving the agent to infer based on the tool name and purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.