qa-mcp

A local MCP server that lets an agent see a running app instead of reading its source.

It opens the app in a real browser, does exactly what it is told, takes pictures, and reports measurements. It makes no decisions: no model calls, no severity scores, no self-healing, no "is this fixed" logic. The calling agent does the judging.

What it gives an agent

Rules measured: horizontal overflow, broken images, missing alt text, tap targets under 24×24, clipped text, plus console errors, page errors and failed requests. Every rule reports its own definition and threshold.

Baseline promotion is deliberately not a tool; accepting a new "correct" look is a human decision and stays in the CLI.

Related MCP server: vouch

Requirements

• Node.js 20 or newer

• Chromium for Playwright: npx playwright install --with-deps chromium

• Optional: electron for the invisible local window engine

Install

npm install qa-mcp
npx playwright install --with-deps chromium

Or from source:

git clone https://github.com/contraliz/qa-mcp.git
cd qa-mcp
npm install
npm run build

Quick start

npx qa-mcp init --url http://localhost:5173/   # writes .qa/config.json + .qa/scenarios/smoke.json
npx qa-mcp doctor                              # node, config, browser, allow-list, artifacts
npx qa-mcp run smoke                           # executes the scenario, writes artifacts

run exits 0 when every step executed, 2 when a step could not (missing selector, timeout). Measurement counts are printed, not judged, so your gate stays your gate.

Register the MCP server

OpenCode (opencode.json):

{
  "mcp": {
    "qa": {
      "type": "local",
      "command": ["npx", "qa-mcp", "serve"],
      "environment": { "QA_HOME": "./qa" }
    }
  }
}

Claude Code (.mcp.json):

{
  "mcpServers": {
    "qa": {
      "command": "npx",
      "args": ["qa-mcp", "serve"],
      "env": { "QA_HOME": "./qa" }
    }
  }
}

Codex (~/.codex/config.toml):

[mcp_servers.qa]
command = "npx"
args = ["qa-mcp", "serve"]
env = { QA_HOME = "./qa" }

Any stdio MCP client works the same way. Without an MCP client, the CLI covers everything except live captures and findings_store.

Scenarios

// .qa/scenarios/checkout.json
{
  "name": "checkout",
  "url": "http://localhost:5173/",
  "steps": [
    { "action": "goto", "value": "/" },
    { "action": "fill", "selector": "#email", "value": "sam@example.com" },
    { "action": "click", "selector": "#place-order" },
    { "action": "wait_for", "selector": "#status" },
    { "action": "capture", "name": "after-order" }
  ]
}

Steps are literal: goto, click, fill, press, hover, select, scroll, wait_for, capture. Capture names are what baseline promote and diff use.

Engines

Under CI=true or QA_HEADLESS=1 the server always uses Playwright headless; Electron needs a desktop session. The Electron helper starts on demand, binds loopback with a per-process token, and exits after 15 idle minutes.

Configuration

.qa/config.json (commit it) with environment variables taking precedence:

{
  "baseUrl": "http://localhost:5173/",
  "engine": "electron",
  "allowHosts": ["localhost:5173", "127.0.0.1:5173"],
  "scenariosDir": ".qa/scenarios",
  "viewport": { "width": 1280, "height": 800 }
}

Safety defaults

• Loopback only. stage_open, scenario_run, goto steps and annotate_start refuse any host outside the allow-list.

• No credentials in URLs. Embedded user:pass@ is rejected; secret-looking query values (token, key, session, …) are masked in stored URLs.

• No writes to your source. Runs land in qa/; the tool never edits code, commits, or promotes baselines.

• Artifacts may contain customer data. Screenshots are of your app. Keep them local and set retention if you publish them.

Storage

qa/
  runs/<run_id>/          meta, steps, measurements, captures, diffs
  findings/<run_id>.json  findings with cited evidence
  annotations/<id>/       annotation.json, crops, full frame
  baselines/<scenario>/   promoted reference frames (commit these)

CLI

qa-mcp serve                          stdio MCP server
qa-mcp init [--url ...] [--engine ...]  write config + a smoke scenario
qa-mcp doctor                         check the machine before blaming the app
qa-mcp run <scenario|name>            CI entrypoint
qa-mcp gc [--keep 20]                 delete the oldest runs
qa-mcp baseline promote|list          human baseline management
qa-mcp --version

Limits

• Web apps only. Native desktop automation is not implemented.

• No visual AI anywhere: "does this look right" is the agent's job.

• Detector set is small and reviewable; contrast and z-order checks are not implemented.

• One browser instance per server; one active scenario at a time.

• Pixel diffs are only comparable across machines inside the pinned browser image (Dockerfile matches the playwright version).

Development

npm run build
npm test          # 21 tests, headless
npm run typecheck
npm run pack:check

Source: src/config.ts (env, config file, allow-list, redaction), src/stage.ts (Playwright engine), src/electron-stage.ts + electron/main.cjs (Electron engine), src/inpage.ts (measurement rules), src/inpage-annotation.ts (annotation overlay), src/scenario.ts, src/diff.ts, src/annotate.ts, src/store.ts, src/server.ts (MCP), src/cli.ts.

License

MIT