Skip to main content
Glama
Sign In
AI QA Agent MCP

AI QA Agent MCP

by sadi-qa
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

Analyze Test Failures

analyze_test_failures

Group failed, timed-out, and flaky Playwright tests from a JSON report by probable failure category to speed up debugging.

Instructions

Analyze failed, timed-out, and flaky tests from an approved Playwright JSON report and group them by probable failure category.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "reportPath": "<reportPath>"
    },
    "name": "analyze_test_failures"
  }
}

Input Schema

NameRequiredDescriptionDefault
reportPathYesPath to the report relative to the approved reports directory, such as json/playwright-results.json.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description is solely responsible for behavioral transparency. It only states that it analyzes and groups, without disclosing output format, prerequisites, side effects, or error behavior. The term 'approved' suggests a constraint but is not explained, leaving a meaningful transparency gap for a tool that produces a categorized analysis.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that packs the core action, input, and outcome without any filler. Every word adds value, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema is present, so the description should explain the return value. It hints at the result ('group them by probable failure category') but does not specify the structure of the output, potential edge cases (e.g., no failures), or how 'approved' is determined. For a tool with one parameter and no nested objects, this is a moderate completeness gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers the single parameter fully with a clear description ('Path to the report relative to the approved reports directory, such as json/playwright-results.json.'), earning a baseline of 3. The description's mention of 'approved Playwright JSON report' aligns with the schema but adds no extra semantic detail beyond what is already provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Analyze'), the resource ('failed, timed-out, and flaky tests from an approved Playwright JSON report'), and the outcome ('group them by probable failure category'). This sharply distinguishes it from sibling tools like list_test_runs or generate_bug_report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you have an approved Playwright report with failures that need categorizing, but it avoids explicit alternatives or exclusions. It doesn't mention when to choose this over generate_bug_report or get_test_run_summary, leaving the agent to infer based on the tool name and purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools