Playwright + local Ollama

This project lets you test a website on your own machine.

For architecture, folder map, how to add a test, and a first-hour walkthrough, see WALKTHROUGH.md.

You get three things:

1. A small login website you can open in the browser

2. Playwright tests that check login, register, and logout

3. Local Ollama that can drive the browser or write a new test from plain English

Nothing is sent to a cloud AI. Ollama and the site both run on your computer.

What you need first

• Node.js 20 or newer

• Ollama installed

Start Ollama and download a model that can use tools:

ollama serve
ollama pull qwen2.5-coder:7b

This repo is already set to use qwen2.5-coder:7b. If you use another model, change OLLAMA_MODEL in .env.

Related MCP server: RunAutomation MCP Server

One-time setup

cp .env.example .env
npm install
npx playwright install chromium

Check that Ollama is ready:

npm run ollama:health

1. Open the login website

npm run site:start

Then open http://127.0.0.1:4173/login.html.

You can sign in with:

You can also create a new account from the register page.

2. Run the tests

When you run tests, Playwright starts the website for you. You do not need to start it first.

npm run test:e2e           # full local-site suite, browser stays open
npm run test:auth          # login, register, logout (faster, headless)
npm run test:auth:headed   # same auth tests, browser visible
npm run test:smoke         # a few quick login checks
npm run test:heal          # prove tests still pass after test IDs change
npm test                   # auth + TodoMVC demo

The headed E2E suite walks through the real site: home redirects, login, register, session reload, logout, and login after test IDs change. You will see one Chrome window. In VS Code use Run and Debug → Playwright: E2E headed or the Playwright: E2E headed task.

Run one file or one test by name:

npx playwright test tests/auth/login.spec.ts
npx playwright test --grep "wrong password"

3. See auto-heal when test IDs change

Tests first look for data-testid (for example login-email).

If you rename that id, the test does not stop. It looks for the field by label, placeholder, or button name. If it still needs help, it asks local Ollama. The new locator is saved in artifacts/healed-locators.json.

Easiest check

npm run test:heal
npx playwright show-report

This runs 3 headed cases. You will see a red LOCATOR FAILED banner, then a green HEALED banner, in the browser and in the terminal. The HTML report includes fail/heal screenshots plus an auto-heal-report.md attachment on each test.

Change IDs for the whole site

npm run site:shift-testids
npm run test:auth
npm run site:reset-testids

Or open the login page and click Toggle remapped test IDs.

To see tests fail after a rename, set AUTO_HEAL=false in .env.

4. Ask Ollama to use the browser

Ollama cannot talk to Playwright by itself. This project connects them.

Type what you want in English:

npm run agent -- "Open http://127.0.0.1:4173/login.html and sign in as admin@local.test / Admin123!"

Start the website first (npm run site:start) so the agent can open it.

A written log of what happened is saved in artifacts/.

To also write a Playwright test from that run:

npm run agent -- "Sign in as admin and open the dashboard" --generate

Or write a test without opening the browser:

npm run generate -- "Sign in as admin@local.test and land on the dashboard"

New tests go to tests/generated/. Read them before you trust them, then run npm test.

Run the same things in VS Code

1. File → Open Folder and choose this project.

2. Install Playwright Test for VS Code when VS Code asks.

Then use any of these:

• Testing (flask icon) — run or debug one test

• Run and Debug — Auth site, auth tests, healing demo, Ollama agent

• Terminal → Run Task — start the site, shift test IDs, run suites

• Explorer → NPM SCRIPTS — every command from package.json

Run with Docker

You can build one image that starts the login site or runs the tests.

npm run docker:build

Start the website (then open http://127.0.0.1:4173/login.html):

npm run docker:site

Run the auth tests inside the image:

npm run docker:test

Or use Compose:

docker compose up auth-lab
docker compose --profile test run --rm tests

Ollama is optional and stays separate:

docker compose --profile ollama up ollama

Headed auto-heal and E2E demos still work best on your machine (npm run test:heal). In Docker they run headless because there is no window.

Useful folders

If something fails

• Ollama is not ready — run ollama serve, then npm run ollama:health

• Website will not open — run npm run site:start and use port 4173

• Browser is missing — run npx playwright install chromium

• Agent talks but does not click — use a model that supports tools, such as qwen2.5-coder:7b