quality-kpi-mcp

Engineering quality KPIs for leadership, on demand: MTTR, DORA metrics, defect escape rate and SLA breaches, answered by your AI assistant from the exports you already have.

quality-kpi-mcp is a Model Context Protocol server that turns incident, deployment and Jira bug exports into the numbers engineering leaders get asked for every month:

"What's our MTTR this quarter, and is it getting better?" "Where do we stand on DORA metrics?" "Which service is causing the most failed deploys?" "Write the monthly quality update for the exec team."

No BI project, no data warehouse, no spreadsheet stitched together the night before the QBR. Drop in three CSV exports, and Claude or Cursor does the analysis.

Demo

1. "Prepare the quality update for my VP" · Claude pulls 90 days of KPIs and writes the executive report. https://github.com/user-attachments/assets/48914ab8-c7f4-4081-9755-02f63e99c92b

2. Root cause and trends · Which service is causing failed deploys, and how MTTR moved month by month.

https://github.com/user-attachments/assets/0b181d09-d29e-4b58-9811-3e5595275f67

3. The leadership dashboard · The HTML report the server exports, top to bottom.

https://github.com/user-attachments/assets/91b2f695-6ea3-46a5-9039-7f0d8287c968

Related MCP server: Koalr

Why this exists

Every quarter I've watched engineering managers spend days assembling the same report: pull incidents from PagerDuty, deploys from CI, bugs from Jira, paste into a spreadsheet, compute MTTR by hand, then write the story. The numbers are always a week stale, and nobody trusts how they were calculated. This server makes the calculation consistent, explainable and instant.

It's the leadership companion to qa-metrics-mcp, which handles test-level triage for engineers.

What it measures

Plus a monthly_quality_review prompt that chains the tools into a VP-ready monthly update.

How the numbers are calculated

DORA tiers use benchmark bands based on the DORA State of DevOps reports (elite: on-demand deploys, lead time under a day, recovery under an hour). The overall tier is the weakest of the four keys. Bands and SLA policy are constants in src/metrics.ts, so you can set your own targets.

What leaders get that a dashboard doesn't

• Findings, not just charts. Rule-based insights call out what changed and why it matters: "payments is a change-failure hotspot: 20% of 54 deploys failed; top cause: schema migration."

• Asks, not just risks. The executive summary ends with concrete next steps an owner can take.

• Any period, any comparison. last_30_days, this_quarter, 2026-09, or any date range, always compared with a sensible baseline.

Works with your existing exports

The loader reads CSV and recognizes common column names from real tools, so most exports work without editing:

Priorities like Highest / Blocker / P1 map to Critical, SEV1 / P1 / critical map to SEV1, and prod / live maps to production. Non-production deploys are ignored. Files are re-read on every call, so dropping in a fresh export updates every answer.

Quick start

git clone https://github.com/Misfit1982/quality-kpi-mcp.git
cd quality-kpi-mcp
npm install          # also compiles TypeScript to dist/
npm run smoke        # end-to-end test over real MCP stdio

The repo ships with six months of synthetic data for a fictional e-commerce company, with real stories planted in it (see Sample data).

Connect to Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "quality-kpi": {
      "command": "node",
      "args": ["/absolute/path/to/quality-kpi-mcp/dist/server.js"],
      "env": { "KPI_DATA_DIR": "/absolute/path/to/your/exports" }
    }
  }
}

Connect to Claude Code

claude mcp add quality-kpi -- node /absolute/path/to/quality-kpi-mcp/dist/server.js

Connect to Cursor

Add the same block to .cursor/mcp.json.

Configuration

Example session

You: Prepare the quality update for my VP. Last 90 days.

Claude: (calls generate_executive_summary, get_change_failure_breakdown, get_defect_aging)

Bottom line: Delivery is rated High on all four DORA keys. Incident recovery improved sharply: MTTR fell 63% to 4.7 hours. Two things need attention: deploy quality on payments and bugs escaping in checkout.

Wins

• MTTR 12.8h → 4.7h after the on-call runbooks rollout

• Customers now find 20% of incidents first, down from 43%

• Lead time for changes halved to 31 hours

Risks and asks

1. payments: 20% of deploys failed, mostly schema migrations. Ask: require tested rollback plans for migrations.

2. checkout: 38% of October's defects were found in production. Ask: add regression coverage before the next release.

3. SHOP-2413 (gift-card orders charged full amount) has been open 43 days against a 2-day SLA. Ask: named owner and fix date this week.

Full generated output: docs/sample-executive-summary.md.

Sample data

Six months (May to October 2026) of synthetic data for a fictional company, ShopLite: about 660 production deploys, 85 incidents and 420 Jira bugs. Planted storylines the tools should find:

1. MTTR drops sharply after on-call runbooks roll out on Aug 1

2. Customer-reported incidents fall after synthetic monitoring goes live, so MTTD drops

3. The payments service's change failure rate spikes in September (a risky schema migration)

4. Defect escape rate climbs in the checkout component

5. A Critical checkout defect sits open for 40+ days, far past its 2-day SLA

6. Lead time for changes improves steadily as PRs get smaller

npm run smoke asserts that every one of them is detected.

Architecture

flowchart LR
  PD[Incident tool<br/>PagerDuty · Opsgenie] -->|incidents.csv| DIR[(Exports folder)]
  CI[CI/CD<br/>GitLab · Actions · Argo] -->|deployments.csv| DIR
  J[Jira] -->|defects.csv| DIR
  DIR --> L[data.ts<br/>CSV loader + column aliases]
  L --> M[metrics.ts<br/>MTTR · DORA · defects · insights]
  M --> S[server.ts<br/>10 MCP tools + review prompt]
  M --> D[dashboard.ts<br/>HTML export]
  S <-->|stdio / MCP| C[Claude · Cursor · any MCP client]

src/
  data.ts        CSV parsing and normalization (severity, priority, environment, column aliases)
  metrics.ts     Pure KPI functions, DORA bands, SLA policy, insights, executive summary
  dashboard.ts   Self-contained HTML leadership dashboard
  server.ts      MCP server: tools, input validation (zod), error handling
scripts/
  generate-sample-data.ts   Deterministic synthetic data with planted storylines
  smoke-test.ts             End-to-end test via the official MCP client

Roadmap

• Pull directly from the Jira, PagerDuty and GitLab APIs instead of CSV exports

• Per-team scorecards and targets

• Scheduled monthly report delivered to Slack or email

• Incident-to-RCA linking with the companion RCA assistant

About

Built by Jared Hood, QA leader with 20 years in SaaS, fintech and media. I build AI-assisted QA and engineering-metrics tooling: MCP servers, test automation, and reporting that turns raw data into decisions.

All data in this repo is synthetic and describes a fictional company.

License: MIT