AI QA Agent MCP

An MCP-enabled QA automation framework built with Playwright and Node.js for executing UI and API automated tests through a Model Context Protocol (MCP) server.

The project demonstrates how traditional test automation can be exposed as reusable MCP tools, allowing an MCP-compatible client to trigger test suites and receive structured test results.

🚀 Features

• Playwright UI automation

• API testing with Playwright

• Page Object Model (POM)

• MCP server integration

• MCP tool-based test execution

• Structured JSON test results

• Login, inventory and checkout test coverage

• Positive and negative test scenarios

• Regression test execution

• Environment-based configuration

• HTML test reporting

• Screenshots on failure

• GitHub Actions CI/CD

• Automated execution on push and pull requests

Related MCP server: mcp-playwright-test

🧰 Tech Stack

• JavaScript

• Node.js

• Playwright

• Model Context Protocol (MCP)

• MCP Inspector

• Git

• GitHub

• GitHub Actions

🏗️ Architecture

                MCP Client / Inspector
                         |
                         v
                    MCP Server
                  (mcp/server.js)
                         |
                         v
                   run_tests Tool
                         |
             +-----------+-----------+
             |                       |
             v                       v
       Test Tool Layer          Suite Selection
       (tools/*.js)        login / inventory /
                           checkout / api /
                           regression
             |
             v
        testRunner.js
             |
             v
          Playwright
         /          \
        v            v
    UI Tests      API Tests
       |
       v
  Page Objects

📁 Project Structure

my-ai-qa-agent/
│
├── .github/
│   └── workflows/
│       └── playwright.yml
│
├── data/
│   ├── customer.js
│   └── users.js
│
├── mcp/
│   └── server.js
│
├── pages/
│   ├── LoginPage.js
│   ├── InventoryPage.js
│   ├── CartPage.js
│   └── CheckoutPage.js
│
├── tests/
│   ├── api/
│   │   └── users.api.spec.js
│   ├── login.spec.js
│   ├── inventory.spec.js
│   └── checkout.spec.js
│
├── tools/
│   ├── testRunner.js
│   ├── runLoginTests.js
│   ├── runInventoryTests.js
│   ├── runCheckoutTests.js
│   ├── runApiTests.js
│   └── runRegressionTests.js
│
├── .env.example
├── .gitignore
├── package.json
├── playwright.config.js
└── README.md

pages/

Contains reusable Page Object Model classes that encapsulate UI locators and user actions.

tests/

Contains Playwright UI and API test specifications.

tools/

Acts as the bridge between MCP requests and Playwright test execution. Individual tools select test suites while testRunner.js executes Playwright and summarizes the results.

mcp/

Contains the MCP server that exposes QA automation capabilities as MCP tools.

.github/workflows/

Contains the GitHub Actions workflow used to execute the automated test suite in CI.

⚙️ Installation

Clone the repository:

git clone https://github.com/bisminizzar84/ai-qa-agent-mcp
cd ai-qa-agent-mcp

Install dependencies:

npm install

Install Playwright browsers:

npx playwright install

Create a local .env file based on .env.example:

BASE_URL=https://www.saucedemo.com

🧪 Running Tests

Run the complete test suite:

npm test

Run tests with a visible browser:

npx playwright test --headed

Run only login tests:

npx playwright test tests/login.spec.js

Run API tests:

npx playwright test tests/api/users.api.spec.js

Open the Playwright HTML report:

npx playwright show-report

🤖 MCP Integration

The project exposes QA automation through an MCP server.

Start the server through MCP Inspector:

npx -y @modelcontextprotocol/inspector@latest node mcp/server.js

The run_tests MCP tool supports multiple suites:

• login

• inventory

• checkout

• api

• regression

Example request:

{
  "suite": "api"
}

Example response:

{
  "suite": "api",
  "status": "passed",
  "total": 1,
  "passed": 1,
  "failed": 0,
  "skipped": 0,
  "durationMs": 1841
}

The MCP layer converts test execution into structured results that can be consumed by MCP-compatible clients.

🔄 CI/CD

GitHub Actions automatically executes the Playwright test suite when code is pushed to main or when a pull request targets main.

The pipeline performs:

1. Repository checkout

2. Node.js setup

3. Dependency installation

4. Playwright browser installation

5. Automated test execution

6. Playwright report upload

This provides automated regression feedback for every code change.

🔮 Future Enhancements

• Connect an LLM to the MCP server for natural-language test execution

• AI-assisted failure analysis

• Automatic defect summaries

• Test generation from natural-language requirements

• Additional API coverage

• Parallel and cross-browser execution

• Dockerized test execution

• Test result notifications