🔍 Allure MCP Analyzer — AI-driven root-cause analysis for test failures

An MCP server + CLI that reads Allure results, clusters failures by normalized error signature (deterministic, no LLM), and then asks an LLM to explain each cluster — product bug, test bug, infrastructure, flaky, or data.

This is the pattern I use to turn a 300-red-test CI run into "3 root causes, here is what to do" in seconds. The clustering step is what makes it cheap and reliable: the LLM analyzes a handful of clusters, not every test.

✨ What's inside

Area	Highlights
Parsing	Reads `allure-results/*-result.json` (status, message, trace, labels, timings)
Clustering	Normalizes volatile tokens (ids, hex, timestamps, paths, numbers) → stable signature → clusters sorted by impact
Root-cause analysis	One LLM call per cluster; strict JSON contract (`category`, `summary`, `confidence`, `evidence`, `suggested_action`) with defensive parsing
Pluggable LLM	`FakeLLM` (offline, deterministic — used in tests & CI without a key) or `AnthropicLLM` (lazy import)
MCP server	Tools: `load_results`, `list_failures`, `cluster_failures`, `analyze_cluster`, `analyze_run`
CLI	`allure-analyze ./allure-results --md report.md --json report.json --top 5` — exit code 1 when failures exist
Reports	Markdown (PR comment / Slack) with a summary table + collapsible affected-tests lists; JSON for dashboards
Agent Skills	`.claude/skills/allure-triage` — a Claude Code playbook that runs the whole triage flow
CI	Lint + tests on 3.11/3.12, demo report uploaded as an artifact

📁 Layout

src/allure_mcp_analyzer/
├── parser.py     # Allure results → TestResult
├── cluster.py    # signature normalization + clustering
├── rca.py        # prompt, LLM call, JSON contract → RootCause
├── report.py     # Markdown / JSON rendering
├── llm.py        # LLM protocol, FakeLLM, AnthropicLLM
├── server.py     # FastMCP server (5 tools)
└── cli.py
examples/allure-results/   # 9 sample results → 3 clusters (assertion, timeout, locator)
tests/                     # unittest / pytest
.claude/skills/            # agent skill for CI triage
.mcp.json                  # register the server with Claude Code / Cursor

🚀 Quick start

pip install -e ".[dev]"            # core + tests
pytest -q                          # 17 tests, all offline

# CLI on the bundled example (FakeLLM when no API key is set)
allure-analyze examples/allure-results --top 3

Real analysis:

pip install -e ".[llm,mcp]"
export ANTHROPIC_API_KEY=...        # see .env.example
export ANTHROPIC_MODEL=...          # https://docs.claude.com/en/docs/about-claude/models
allure-analyze ./allure-results --md report.md

🤖 As an MCP server (Claude Code / Cursor)

.mcp.json is already in the repo — from the repo root:

claude
> Use allure-analyzer to triage ./allure-results and tell me which clusters block the release.

Or invoke the bundled skill directly: /allure-triage.

🧠 How clustering works

"AssertionError: expected total 59.98 but got 49.99"
"AssertionError: expected total 120.00 but got 100.00"
        ↓ normalize()
"AssertionError: expected total <n> but got <n>"     → one cluster, 3 tests

Timeouts on a shared dependency collapse the same way, so an outage shows up as one infrastructure cluster instead of fifty unrelated reds.

🔌 CI integration

The workflow uploads report.md / report.json as an artifact on every run. To post the Markdown as a PR comment, pipe it through gh pr comment --body-file report.md.

📄 License

MIT