<div align="center">

# ci-triage-mcp

### Ask Claude why your last CI run failed. Get a real answer built from parsed logs, not a guess from raw text.

![Node.js](https://img.shields.io/badge/Node.js-18%2B-6552D0?style=flat-square&logo=nodedotjs&logoColor=white)
![TypeScript](https://img.shields.io/badge/TypeScript-6552D0?style=flat-square&logo=typescript&logoColor=white)
![MCP](https://img.shields.io/badge/MCP-Server-6552D0?style=flat-square)
![GitHub Actions](https://img.shields.io/badge/GitHub%20Actions-6552D0?style=flat-square&logo=githubactions&logoColor=white)

**[Pipeline](#pipeline) · [Guardrail](#guardrail) · [Setup](#setup) · [Tools](#tools) · [Extractors](#extractors) · [Dashboard](#dashboard)**

</div>

This is an [MCP](https://modelcontextprotocol.io) server for Claude Desktop and Claude Code. You
ask, in conversation, why a GitHub Actions run failed; Claude fetches the run, and this server
deterministically extracts the actual failure signal -- test name, class, assertion message, stack
trace -- from JUnit/Surefire XML, ESLint, tsc, Prettier, or Maven console output. Claude never
reads a raw log to guess what went wrong; it reasons over structured data a real parser produced.

No API key, no server to deploy, nothing running unattended. The "LLM" is whichever Claude client
is already open when you ask -- same design as [k6-loadtest-mcp](https://github.com/krishanchawla/k6-loadtest-mcp).

## Why this exists instead of just pasting a log into Claude

> The LLM plans and explains. Code fetches and parses. The two never trade places.

Pasting a 2,000-line CI log into a chat works, but it wastes context on install/browser-download
noise and leaves the model guessing at which lines actually matter. This server does the
mechanical part in real parsers -- JUnit/Surefire XML via a proper XML parser, not regex on a raw
log wherever structured output exists -- so what Claude sees is already reduced to the handful of
lines that are the failure. It also remembers: a local history store means "this test has failed
the same way 3 times this month" is something the tool tells you, not something you have to recall.

<details>
<summary><b>Example</b> — what asking this looks like in conversation</summary>
<br>

```
You:    Why did the last run of playwright-typescript-framework fail on main?

Claude: Fetched run 30298934612, extracted 14 failure signals across the run's failed
        shards, and cross-referenced them against local history...

        This run mixes one real bug with a batch of environment flakiness:

        🔴 Real bug -- axe-core flags a WCAG 2 AA color-contrast violation on the
           products, cart, and checkout pages (3 signals, one shared breadcrumb
           component: #6d68cf on #0a0a11, a 4.25:1 ratio against the 4.5:1 minimum).

        🟡 Not a bug -- 11 more signals (API timeouts, cancelled page loads) across
           unrelated specs all point at one shard losing connectivity to
           playground.krishanchawla.com mid-run, not broken tests.

        Fix the breadcrumb's text-accent token once; rerun the flaky shard once the
        target's back up.
```

Condensed, but the real shape of it -- this is an actual run this project's own live-testing was
validated against (see [Extractors](#extractors)), not a hypothetical.

</details>

## Pipeline

```mermaid
flowchart TD
    A["you: 'why did the last run of\nplaywright-typescript-framework fail?'"] --> B[fetch_pipeline_run]
    B --> C{artifact named\njunit/surefire?}
    C -->|yes| D[download + unzip artifact\nparse JUnit/Surefire XML]
    C -->|no, or expired| E[get_job_log\nparse ESLint / tsc / Prettier / Maven console]
    D --> F[extract_failure_signal\nstructured FailureSignal + signature]
    E --> F
    F --> G[find_similar_past_failures\nlocal history lookup]
    G --> H[Claude writes the explanation\nfrom structured data]
    H -. optional .-> I[record_triage_note\nlocal only, no confirmation needed]
    H -. optional, ask first .-> J[publish_triage\n→ shared dashboard]

    G1["Guardrail: allowedRepos,\nnot agent-editable"]
    B -. enforced before every fetch .-> G1

    style G1 fill:#6552D0,color:#fff,stroke:#333
    style J stroke-dasharray: 4 3
```

`triage_pipeline_failure` chains fetch → download/parse every relevant artifact and failed job's
log → history lookup, in one call, and falls back to job-log parsing per artifact rather than
aborting the whole run if one has expired. The granular tools exist for targeting one specific job.

## Guardrail

Actions data (runs, job logs, artifacts) is only ever fetched for repos listed in `allowedRepos`
in `~/.ci-triage-mcp/config.json` (empty by default). **The tools cannot add to this list
themselves** -- the same shape as k6-loadtest-mcp's host allowlist: an agent-authored call,
legitimate or prompt-injected, doesn't get to expand its own blast radius. Add a repo yourself
once you've confirmed you're authorized to read its Actions data:

```json
{ "allowedRepos": ["krishanchawla/playwright-typescript-framework", "krishanchawla/selenium-java-framework"] }
```

## Setup

Prerequisites: Node.js 18+, and a GitHub token available -- either the `GITHUB_TOKEN` env var, or
the [gh CLI](https://cli.github.com/) already logged in (`gh auth login`); this server falls back
to `gh auth token` automatically. This is *your own* local credential, used to call GitHub's API
on your own behalf -- nothing is ever stored server-side or embedded in a deployed service, which
is deliberate (see [Why not just call an LLM API directly](#why-not-just-call-an-llm-api-directly)).

```bash
npm install
npm run build
```

### Try the extractors locally first

```bash
npm run harness   # runs every parser against fixtures/ and checks the counts -- no GitHub calls
```

### Try it against a real repo, without going through MCP

```bash
npm run live-check <owner> <repo> [branch]   # defaults to krishanchawla/playwright-typescript-framework main
```

Requires a real token (`GITHUB_TOKEN` or `gh auth login`). Exercises the same fetch → extract →
history-match logic `triage_pipeline_failure` wires together, printed directly instead of over MCP
transport -- useful for checking a parser against a real log before trusting it in conversation.
This is how every bug documented in [Extractors](#extractors) below was actually found.

Both `playwright-typescript-framework` and `selenium-java-framework` also have a standing `demo`
branch (their `main` branches stay clean, ready-to-clone framework skeletons -- see each repo's own
README) that exists specifically to give this project real, current CI failures to test against,
instead of hoping `main`'s last 30 runs happen to include one:

```bash
npm run live-check krishanchawla playwright-typescript-framework demo
```

### Register with Claude Desktop / Claude Code

**Claude Code**, from a terminal:

```bash
claude mcp add ci-triage-mcp -- node /absolute/path/to/ci-triage-mcp/dist/index.js
```

If `GITHUB_TOKEN` isn't already in your shell environment and you're not relying on `gh auth
token`, set it at registration time instead of in your current shell -- the server won't see a
variable set afterward in some other terminal:

```bash
claude mcp add ci-triage-mcp -e GITHUB_TOKEN=<token> -- node /absolute/path/to/ci-triage-mcp/dist/index.js
```

**Claude Desktop**, edit `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "ci-triage-mcp": {
      "command": "node",
      "args": ["/absolute/path/to/ci-triage-mcp/dist/index.js"]
    }
  }
}
```

**Fully quit and restart Claude Desktop/Claude Code after registering or changing this** -- it
spawns the server once at startup and won't notice config/env changes made afterward, including a
rebuilt `dist/`. This bites people (it bit me while building this) far more often than it should.

Then add the repos you want triaged to `allowedRepos` (see [Guardrail](#guardrail)), and
ask, e.g.:

> Why did the last run of playwright-typescript-framework's CI fail on main?

## Tools

| Tool | Purpose |
|---|---|
| `fetch_pipeline_run` | Resolve a run (by ID, or latest failure on a branch) -> jobs + artifacts |
| `extract_failure_signal` | One job/artifact -> structured `FailureSignal[]`, real parsers only |
| `find_similar_past_failures` | Read-only local history lookup by signature |
| `record_triage_note` | Persist your explanation to local history (no confirmation needed -- local file only) |
| `triage_pipeline_failure` | All of the above chained for a whole run |
| `publish_triage` | Publish to a [dashboard](#dashboard) -- not deployed publicly yet, works against a self-hosted instance |

## Extractors

| Source | Parser | Used for |
|---|---|---|
| JUnit / Surefire XML | real XML parser (`fast-xml-parser`) | any repo that uploads `*.xml` test-report artifacts |
| Playwright `list` reporter console output | line-pattern parser, **validated against real logs** | `playwright-typescript-framework`'s sharded test jobs once their `junit-results` artifact has expired (14-day retention) -- see [live-check](#try-it-against-a-real-repo-without-going-through-mcp) |
| ESLint stylish (`eslint .`'s default output) | line-pattern parser | `playwright-typescript-framework`'s lint job |
| tsc (`tsc --noEmit`'s default output) | line-pattern parser | same lint job's type-check step |
| Prettier (`prettier --check`) | line-pattern parser | same lint job's format-check step |
| Maven/Surefire console "Results" block | line-pattern parser, **validated against a real log** | `selenium-java-framework`, which doesn't currently upload `target/surefire-reports/` as an artifact -- see [Roadmap](#roadmap) |

`selenium-java-framework` has never had a failed CI run on its own -- there was nothing real to
validate the Maven console parser against, so it shipped tested only against a hand-written
fixture. To actually check it rather than leave that as a guess: a throwaway branch with one
assertion deliberately flipped (`$39.50` → `$999.99`), opened as a PR (triggers the same workflow,
touches `main`'s history not at all), triaged for real once it failed, then closed unmerged. The
parser correctly pulled the AssertJ diff
(`expected:<"$[999.99]"> but was:<"$[39.50]">`), test name, class, and line straight out of the
real console output on the first try -- see [PR #1](https://github.com/krishanchawla/selenium-java-framework/pull/1)
(closed) for the actual run this validated against.

Five bugs live-testing against real `playwright-typescript-framework` runs has actually surfaced,
in the order they were found:

- **GitHub prefixes every line of a raw job log with an ISO-8601 timestamp** (stripped once at the
  source in `github.ts`, `getJobLog`). If you add a new text-based parser, write its regexes
  against already-stripped content -- every fixture in `fixtures/` is pre-stripped for exactly this
  reason.
- **A `expect(x).toEqual(y)` failure against a large object opens with a pretty-printed JSON dump**
  before anything readable -- `parsePlaywrightList` prefers the annotated `> N | expect(...)`
  source line instead when the message would otherwise just be a bare `Error: [`.
- **Playwright retries re-print the same failure block.** A test that retries twice re-dumps the
  same (sometimes huge) error text three times into what `parsePlaywrightList` treats as one
  failure's block -- one accessibility assertion against a large violations object produced a
  ~60KB `stackTrace` on a single signal this way. `src/extract/truncate.ts` caps every extracted
  `stackTrace` at 4000 chars now, in every parser, not just this one.
- **An artifact GitHub still lists (with a real file size) can still 410 on download** once it's
  past its retention window -- `listArtifacts` doesn't reflect expiry, only the download attempt
  does. `triage_pipeline_failure` now catches each artifact's download individually and falls back
  to job-log parsing for that job instead of aborting the whole call.
- **The bare-JSON-opener fix above only ever applied to `parsePlaywrightList`, not
  `parseJUnitXml`.** Playwright's own JUnit reporter truncates a `<failure message="...">`
  attribute the exact same way its list reporter's first line gets truncated -- so the *preferred*
  path (real XML) was producing a worse message (`"["`) than the *fallback* path (scraped console
  text) for the identical failure. The message-picking logic is now shared (`src/extract/message.ts`)
  so the two parsers can't drift on this again.

## Why not just call an LLM API directly

Because that would mean an Anthropic API key living on a public-facing server, paid for per call
and reachable if that server is ever compromised -- a materially different (and worse) risk than
anything else in this project. This server never calls an LLM API at all: it's tool calls that the
*already-running* Claude Desktop/Code session decides to make, under whatever plan you're already
paying for. Nothing here would need to change if you're using Claude Free, Pro, or Max -- the
server doesn't know or care.

## Dashboard

`dashboard/` is an optional Spring Boot + Thymeleaf app, sibling to `k6-loadtest-mcp`'s own
`dashboard/`, that `publish_triage` posts a triage result to -- gives it a real, shareable URL
instead of living only in one Claude conversation. Same design as the load-test dashboard: a
plain jar with its own embedded server, H2 file-backed storage, bearer-token-gated ingest
separate from HTTP-Basic-gated (or public-demo, unauthenticated) viewing.

What it adds beyond just listing runs:
- **Category breakdown chart** across every extracted signal, not just each run's headline
  category -- a single run routinely mixes categories (the [Example](#why-this-exists-instead-of-just-pasting-a-log-into-claude)
  above is real: one CI run produced a genuine accessibility regression *and* an unrelated cluster
  of infra timeouts, and counting at the signal level is the only way that doesn't get hidden
  behind whichever one happened to run first).
- **Recurrence tracking** -- every signal is matched against prior triage runs for the same repo
  by its stable `signature`; the detail page shows "seen N× before" instead of treating every
  failure as novel, and the list page surfaces a standing "Recurring failures" panel.
- **Narrative-first detail page** -- the LLM's explanation and suggested fix are the headline
  content, with raw stack traces behind a `<details>` disclosure per signal, not the other way
  around.
- **A 14-day run-volume sparkline** per repo, so a rising or falling triage rate is visible at a
  glance, not just a bare run count.

<p align="center">
  <img src="docs/screenshot-detail.jpg" width="49%" alt="Triage run detail page, showing three separate stories in one run" />
  <img src="docs/screenshot-repo.jpg" width="49%" alt="Repo dashboard with category breakdown and real recurring failures" />
</p>
<p align="center"><sub>Two real triages of <code>playwright-typescript-framework</code>'s <code>demo</code> branch (see
<a href="#try-it-against-a-real-repo-without-going-through-mcp">live-testing</a>), published to a locally-run instance --
no public instance is deployed yet (see <a href="#roadmap">Roadmap</a>). The WCAG contrast bug recurred organically
between the two, unprompted -- the "×2" and "seen 1× before" badges are real, not staged.</sub></p>

### Build and run it locally

```bash
cd dashboard
mvn -q package                       # -> target/ci-triage-dashboard.jar
DASHBOARD_API_TOKEN=<pick-a-token> java -jar target/ci-triage-dashboard.jar
```

Then point `dashboardUrl` in `~/.ci-triage-mcp/config.json` at it (e.g.
`"http://localhost:8081"` while testing locally) and set `CI_TRIAGE_DASHBOARD_TOKEN` to match, on
the MCP server's own registration (see [Setup](#setup) for why it has to be set there, not a
shell env var).

### Deploying it

Same posture as `k6-loadtest-mcp`'s dashboard -- a self-contained jar with its own embedded
server (Spring Boot 4 / Jakarta EE, needs Tomcat 11+ if you ever did drop it into an external
container, which there's no reason to). Run it via systemd with:

| Env var | Required | Purpose |
|---|---|---|
| `DASHBOARD_API_TOKEN` | yes, to accept triage results | Bearer token `publish_triage` must send. Ingest returns 503 until set. |
| `DASHBOARD_BASIC_AUTH_USER` / `DASHBOARD_BASIC_AUTH_PASS` | no | HTTP Basic guarding every page except `/api/**`. Set both for a private/gated dashboard (the default posture for your own real data); leave `PASS` unset for the public-demo posture (reads open, same as the load-test dashboard). |
| `DASHBOARD_PUBLIC_BASE_URL` | yes, for correct links | Externally visible base URL used to build the shareable links `publish_triage` returns. |
| `DASHBOARD_DEMO_ALLOWED_REPOS` | no | Public-demo-mode only: comma-separated `owner/repo` allowlist for the ingest endpoint, once the bearer token is effectively public. Self-host default (unset) accepts any repo -- `allowedRepos` on the MCP side has already gated what could be published in the first place. |
| `DASHBOARD_RETENTION_DAYS` | no | Public-demo-mode only: auto-prune triage runs older than N days. Unset keeps everything forever. |
| `DASHBOARD_PORT` | no (default `8081`) | Port the embedded server listens on -- deliberately different from the load-test dashboard's `8080` default so both can run on the same box without a collision. |

## Roadmap

- **Artifact upload for `selenium-java-framework`.** The console parser works (see
  [Extractors](#extractors)), but a `target/surefire-reports/` upload-artifact step (mirroring what
  `playwright-typescript-framework` already does) would let it use the real JUnit XML parser
  instead -- structured XML over scraping console output whenever it's available at all.
- **CI on this repo itself.** `npm run harness` runs the extractor fixtures locally but nothing
  runs it on push -- a `.github/workflows` job that fails loudly on a broken parser would be a
  cheap, honest thing for a CI-triage tool to be missing.
- **Actually deploy the dashboard** to the VPS and wire `dashboardUrl` there -- built, verified
  locally, and now proven end-to-end against a real triage result (see the [Dashboard](#dashboard)
  screenshots above), just not yet live anywhere public.
- **`live-check.ts` and `triage_pipeline_failure` reimplement the same fetch → extract pipeline
  independently.** They drifted once already -- `live-check.ts` already caught per-artifact
  download failures individually, but `triage_pipeline_failure` didn't until a live test against
  the actual MCP tool caught the gap. Worth factoring into one shared function both call, so a fix
  to one can't silently miss the other again.

---

<div align="center">
<sub>Built by <a href="https://github.com/krishanchawla">Krishan Chawla</a> · <a href="https://krishanchawla.com">krishanchawla.com</a></sub>
</div>