Automation Coverage MCP

Coverage-driven test generation for the RHDH plugin forest. This MCP analyzes git changes, reads Istanbul/LCOV numbers, and tells agents which tests to write at which layer. UI work is handed to Playwright MCP as explore-then-write briefs — this server does not drive a browser.

Plugins live at multiple levels (workspaces/<ws>/plugins/<plugin>, backend modules, overlay e2e, RHDH product e2e). Discovery walks that layout instead of hardcoding package names.

What it does

1. Discover packages across rhdh-plugins, community-plugins, rhdh, rhdh-plugin-export-overlays, and shared Playwright helper repos.

2. Analyze the current git diff (branch vs origin/main plus uncommitted work) and classify each file (util, React page, backend router, overlay metadata, platform, …). When the branch is clean, pass mode=workspace to scan src instead.

3. Read coverage (coverage/coverage-final.json or lcov.info) from the cwd / scoped packages only (never a sibling workspace) and intersect uncovered lines with the diff.

4. Recommend layers using the cheapest-layer-wins ladder (unit → integration → component → plugin Playwright → overlay/cluster). Layers are pluggable YAML.

5. Emit briefs agents can execute. UI briefs are Playwright MCP prompts: navigate, snapshot, generate locators, verify, then write the spec.

Coverage ranks gaps. It is not a merge gate. Each brief states the failure the test must catch.

Related MCP server: Quality MCP

Pair with Playwright MCP

Add both servers to Cursor MCP config (~/.cursor/mcp.json, or .cursor/mcp.json in this clone). Playwright needs the testing capability so browser_generate_locator and browser_verify_* are available.

Substitute <ABS_PATH_TO_THIS_CLONE> with the absolute path of this repository on your machine (the directory that contains src/index.ts).

{
  "mcpServers": {
    "automation-coverage": {
      "command": "npx",
      "args": ["tsx", "<ABS_PATH_TO_THIS_CLONE>/src/index.ts"]
    },
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest", "--caps=testing"]
    }
  }
}

If the Cursor workspace is this repository, you can use the relative entry already in .cursor/mcp.json (./src/index.ts) instead of an absolute path.

Agent loop for UI gaps:

generate_playwright_brief → Playwright browser_navigate → browser_snapshot → interact → browser_generate_locator → browser_verify_* → write @playwright/test spec mirroring a neighbor → run until green.

Do not write Playwright from the scenario paragraph alone.

Tools

Prompts: fill_automation_gaps, playwright_from_coverage, unit_from_coverage.

Pluggable layers

Builtins (from the RHDH test-placement ladder):

Add or override layers in .automation-coverage.yaml (see config/rhdh-forest.example.yaml):

disabledLayers: [cluster-e2e]
layers:
  - id: contract
    title: HTTP contract
    kind: custom
    cost: s
    fileKinds: [backend-router]
    testGlobs: ["**/*.contract.test.ts"]
    generator: Write a contract test against the public HTTP schema.

Config lookup

1. AUTOMATION_COVERAGE_CONFIG (absolute path to a YAML file)

2. .automation-coverage.yaml walking up from cwd

3. ~/.config/automation-coverage/config.yaml

4. Auto-detect: walk up from cwd until a parent directory contains sibling clones named rhdh-plugins, community-plugins, rhdh, rhdh-plugin-export-overlays, lightspeed-playwright-e2e, and/or backstage

Forest paths in YAML expand ~ and ${ENV_VAR}. Copy config/rhdh-forest.example.yaml and substitute ${FOREST_ROOT} (or export it) — that value is the parent directory that contains your clones, not rhdh-plugins itself.

Omit forest entries you have not cloned. If ${FOREST_ROOT} is left unexpanded, those rows are ignored and auto-detect is used.

Run

cd <ABS_PATH_TO_THIS_CLONE>
npm install
npm test
npm start   # stdio MCP

Produce a coverage report in the plugin workspace before planning (yarn test:all / backstage-cli repo test --coverage). If no report is present, changed lines are treated as uncovered.

Agent workflow

1. generate_test_plan with cwd set to the plugin workspace. Use mode=workspace when git diff is empty (existing plugin, not a feature branch).

2. Implement every playwrightMcp: false item by mirroring template

3. For playwrightMcp: true items, run generate_playwright_brief and drive Playwright MCP

4. Skip files in skipped (ignored wiring, already covered, non-source, already has a neighbor test)