# AI QA Agent MCP

An MCP-enabled QA automation framework built with Playwright and Node.js for executing UI and API automated tests through a Model Context Protocol (MCP) server.

The project demonstrates how traditional test automation can be exposed as reusable MCP tools, allowing an MCP-compatible client to trigger test suites and receive structured test results.

## 🚀 Features

- Playwright UI automation
- API testing with Playwright
- Page Object Model (POM)
- MCP server integration
- MCP tool-based test execution
- Structured JSON test results
- Login, inventory and checkout test coverage
- Positive and negative test scenarios
- Regression test execution
- Environment-based configuration
- HTML test reporting
- Screenshots on failure
- GitHub Actions CI/CD
- Automated execution on push and pull requests

## 🧰 Tech Stack

- JavaScript
- Node.js
- Playwright
- Model Context Protocol (MCP)
- MCP Inspector
- Git
- GitHub
- GitHub Actions

## 🏗️ Architecture

```text
                MCP Client / Inspector
                         |
                         v
                    MCP Server
                  (mcp/server.js)
                         |
                         v
                   run_tests Tool
                         |
             +-----------+-----------+
             |                       |
             v                       v
       Test Tool Layer          Suite Selection
       (tools/*.js)        login / inventory /
                           checkout / api /
                           regression
             |
             v
        testRunner.js
             |
             v
          Playwright
         /          \
        v            v
    UI Tests      API Tests
       |
       v
  Page Objects
```

## 📁 Project Structure

```text
my-ai-qa-agent/
│
├── .github/
│   └── workflows/
│       └── playwright.yml
│
├── data/
│   ├── customer.js
│   └── users.js
│
├── mcp/
│   └── server.js
│
├── pages/
│   ├── LoginPage.js
│   ├── InventoryPage.js
│   ├── CartPage.js
│   └── CheckoutPage.js
│
├── tests/
│   ├── api/
│   │   └── users.api.spec.js
│   ├── login.spec.js
│   ├── inventory.spec.js
│   └── checkout.spec.js
│
├── tools/
│   ├── testRunner.js
│   ├── runLoginTests.js
│   ├── runInventoryTests.js
│   ├── runCheckoutTests.js
│   ├── runApiTests.js
│   └── runRegressionTests.js
│
├── .env.example
├── .gitignore
├── package.json
├── playwright.config.js
└── README.md
```

### `pages/`
Contains reusable Page Object Model classes that encapsulate UI locators and user actions.

### `tests/`
Contains Playwright UI and API test specifications.

### `tools/`
Acts as the bridge between MCP requests and Playwright test execution. Individual tools select test suites while `testRunner.js` executes Playwright and summarizes the results.

### `mcp/`
Contains the MCP server that exposes QA automation capabilities as MCP tools.

### `.github/workflows/`
Contains the GitHub Actions workflow used to execute the automated test suite in CI.

## ⚙️ Installation

Clone the repository:

```bash
git clone https://github.com/bisminizzar84/ai-qa-agent-mcp
cd ai-qa-agent-mcp
```

Install dependencies:

```bash
npm install
```

Install Playwright browsers:

```bash
npx playwright install
```

Create a local `.env` file based on `.env.example`:

```env
BASE_URL=https://www.saucedemo.com
```

## 🧪 Running Tests

Run the complete test suite:

```bash
npm test
```

Run tests with a visible browser:

```bash
npx playwright test --headed
```

Run only login tests:

```bash
npx playwright test tests/login.spec.js
```

Run API tests:

```bash
npx playwright test tests/api/users.api.spec.js
```

Open the Playwright HTML report:

```bash
npx playwright show-report
```

## 🤖 MCP Integration

The project exposes QA automation through an MCP server.

Start the server through MCP Inspector:

```bash
npx -y @modelcontextprotocol/inspector@latest node mcp/server.js
```

The `run_tests` MCP tool supports multiple suites:

- `login`
- `inventory`
- `checkout`
- `api`
- `regression`

Example request:

```json
{
  "suite": "api"
}
```

Example response:

```json
{
  "suite": "api",
  "status": "passed",
  "total": 1,
  "passed": 1,
  "failed": 0,
  "skipped": 0,
  "durationMs": 1841
}
```

The MCP layer converts test execution into structured results that can be consumed by MCP-compatible clients.

## 🔄 CI/CD

GitHub Actions automatically executes the Playwright test suite when code is pushed to `main` or when a pull request targets `main`.

The pipeline performs:

1. Repository checkout
2. Node.js setup
3. Dependency installation
4. Playwright browser installation
5. Automated test execution
6. Playwright report upload

This provides automated regression feedback for every code change.

## 🔮 Future Enhancements

- Connect an LLM to the MCP server for natural-language test execution
- AI-assisted failure analysis
- Automatic defect summaries
- Test generation from natural-language requirements
- Additional API coverage
- Parallel and cross-browser execution
- Dockerized test execution
- Test result notifications