# QA Copilot MCP Server

A [Model Context Protocol](https://modelcontextprotocol.io) server that hands an AI
assistant a small set of concrete QA/test-intelligence tools instead of asking it to
guess: **summarize a Playwright run**, **find flaky tests across several runs**,
**explain a failure heuristically**, **scaffold a Playwright test stub**, and **read a
GitHub repo's recent CI status**. Point Claude Desktop, Claude Code, or any other
MCP-compatible host at it and those become real tool calls, not free-text guesses.

It's the "give the AI real data, not vibes" piece of a QA automation portfolio: the
other repos in this portfolio (`healthcare-qa-automation-framework`,
`healthcare-data-quality-framework`) produce Playwright JSON reports and CI runs; this
server is what turns those into something an assistant can actually query.

## Why it's built this way

- **The protocol wiring is a thin layer over plain functions.** Every tool's real logic
  lives in `src/playwright/`, `src/github/`, and `src/tools/` as small, pure(-ish),
  independently unit-tested modules. `src/index.ts` only registers them with the MCP
  SDK - so the hard-to-get-wrong part (report parsing, flaky detection, heuristics) is
  covered by fast unit tests, and the protocol wiring is covered by a separate
  end-to-end smoke test (below).
- **GitHub access is pluggable, the same pattern as the LLM integrations elsewhere in
  this portfolio.** `GitHubClient` is an interface; `RealGitHubClient` wraps `fetch`
  against the GitHub REST API, `FakeGitHubClient` returns canned data. That makes
  `get_repo_ci_status` fully unit-testable offline, while the real client is genuine,
  runnable code - an MCP host running on a developer's machine has normal internet
  access, even in environments (like this repo's own CI sandbox) that don't.
- **Test-stub generation is deterministic, not LLM-based, on purpose.** Given a named
  sequence of `data-testid` interactions, `generate_playwright_test_stub` always
  produces the same output - instant, free, reproducible. A different tool for a
  different job (generating test *cases* from a spec or user story, where an LLM
  actually adds value) is a separate project in this portfolio.
- **A protocol-level smoke test, not just unit tests.** `smoke-test.mjs` spawns the
  built server as a real child process and drives it with the official MCP `Client`
  over stdio - `tools/list`, then a `tools/call` per tool - so CI proves the actual
  wire protocol works, not just the functions behind it.

## Tools

| Tool | What it does |
| --- | --- |
| `summarize_test_run` | Parses a Playwright JSON reporter output into pass/fail/timeout/skipped/flaky counts, duration, and full failure details. |
| `list_flaky_tests` | Compares the same tests across 2+ Playwright JSON reports and flags any whose outcome wasn't consistent. |
| `explain_failure` | Heuristic, pattern-matched root-cause guess for a raw error message. Fully offline. |
| `generate_playwright_test_stub` | Deterministically scaffolds a runnable Playwright test from a named sequence of `data-testid` interactions. |
| `get_repo_ci_status` | Fetches a public GitHub repo's recent Actions runs and summarizes pass/fail/in-progress counts. |

## Architecture

```
src/
  index.ts                    MCP server: registers all 5 tools, connects over stdio
  playwright/reportParser.ts  Pure Playwright-JSON parsing: summarizeReport, findFlakyTests
  github/githubClient.ts      GitHubClient interface + RealGitHubClient (fetch) + FakeGitHubClient
  tools/
    explainFailure.ts         Heuristic pattern-matching (ported from the triage_failure.py
                               script in healthcare-qa-automation-framework)
    generateTestStub.ts       Deterministic Playwright test-file template generator
    getRepoCiStatus.ts        Wraps GitHubClient into pass/fail/in-progress counts
test/                         Vitest unit tests for every module above
fixtures/                     Sample Playwright JSON reports (3 runs, used for flaky-test tests)
smoke-test.mjs                End-to-end MCP protocol smoke test (real child process, real client)
.github/workflows/ci.yml      Type-check, unit tests, build, protocol smoke test
```

## Tech stack

TypeScript · Node.js · `@modelcontextprotocol/sdk` · Zod · Vitest · GitHub REST API ·
GitHub Actions

## Running it locally

```bash
npm install
npm run build       # compiles to dist/
npm test            # unit tests (27 tests, fully offline)
npm run smoke        # builds, then drives the real server over stdio via the MCP client
```

### Using it from an MCP host

Point any MCP-compatible client at the built server, e.g. in Claude Desktop's
`claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "qa-copilot": {
      "command": "node",
      "args": ["/absolute/path/to/qa-copilot-mcp-server/dist/index.js"]
    }
  }
}
```

Set `GITHUB_TOKEN` in the environment to raise `get_repo_ci_status`'s unauthenticated
GitHub rate limit (60 requests/hour without one, 5,000/hour with one) - it works fine
without a token for occasional use.

## CI

Every push and pull request to `main` runs a single job: type-check
(`tsc --noEmit`), the unit test suite (`vitest run`), a production build, and the
protocol-level smoke test (`node smoke-test.mjs`) against the built output. No network
access is assumed or required anywhere in CI - `get_repo_ci_status` is exercised only
against `FakeGitHubClient` in the unit tests, and the smoke test treats a network-free
`get_repo_ci_status` result as a pass as long as the server reports it cleanly via MCP's
`isError` channel instead of crashing.

## What this is not

This isn't a general-purpose GitHub or CI dashboard - it's a small, focused set of tools
built specifically around the Playwright-report and GitHub-Actions shapes this
portfolio's other repos actually produce, meant to demonstrate how an AI assistant gets
wired up with real, verifiable test data instead of being asked to reason about test
results from a paraphrase.