playwright-e2e-mcp
Runs Playwright end-to-end tests against Firefox: the browser argument selects Firefox (matched against the Playwright config project names), and Firefox can be launched headed or headless for test runs, failure diagnoses, live DOM inspection (inspect-page/validate-selector) and visual comparisons.
Reads the project's Git working tree (git status, falling back to git diff HEAD~1 and recent mtimes) to discover the agent's recent changes and extract the locators those files actually declare, so generated Playwright tests are built from the project's real selectors.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@playwright-e2e-mcprun my checkout e2e tests and tell me why they failed"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
playwright-e2e-mcp
An MCP server that diagnoses and fixes failing Playwright tests. It is the layer that sits next to a browser-driving MCP: Playwright MCP gives an agent hands in the browser; this server gives it a test runner, a failure analyst and a verified fixer.
run-test ──▶ get-failure ──▶ suggest-fix (apply + re-run) ──▶ green
│ │ └─ refuses when the element is really gone (app bug)
│ └─ DOM at failure, failed requests, console errors
└─ background runs, progress ──▶ analyze-history (flaky / broken / regressed)What it does that a browser-driving MCP or a raw npx playwright test does not:
Structured runs. Stats plus each failure's
file:line, kind (assertion, timeout, dead dev server, syntax error, crash, full disk…) and a concrete fix hint. Long suites run in the background with live progress.Diagnosis from the trace. The DOM at the moment of failure, the failed action, the 4xx/5xx requests and console errors, read from Playwright's own trace.
Fixes that are proven, not guessed.
suggest-fixmatches a broken locator against the DOM at failure, swaps in a Playwright locator proven unique on the page (getByRole('button', { name: 'Proceed to checkout' })), re-runs the test and reverts the edit if it is still red. When nothing on the page plausibly matches, it says the element is missing and does not patch the test to hide a real bug.Flakiness across runs. Every run is recorded locally;
analyze-historyseparates tests that flip (flaky) from ones that started failing (regressed) or always fail (broken), and groups failures that share one cause.Pages behind a login. Every page tool accepts a Playwright
storageState, extra headers, and click/fill steps to reach a modal or a later step.Small on the wire. In the benchmark, diagnosing and fixing one drifted locator cost about 5,300 tokens here against about 7,400 with Playwright MCP plus a shell, and the fix was applied and verified without the agent editing anything.
What it does not do
It does not drive an exploratory browsing session step by step, and it does not replace Playwright's own Test Agents (planner, generator, healer) inside VS Code. Use Playwright MCP for interactive browsing and this server for running, diagnosing and fixing the suite; they work side by side in the same client.
Demo
Hosted endpoint — what an initialize + tools/list round-trip against
https://playwright-e2e-mcp.vercel.app/api/mcp returns for a client that sends the
bearer token (without one, only list-tests and get-failure are listed):

A real test run — run-test served over stdio by npx -y playwright-e2e-mcp
against the bundled examples/sample-test.spec.ts (actual output, unedited):

Images are rendered with node scripts/gen-demo-images.mjs: the run-test card is real
captured output; the endpoint card is an illustration of the authenticated listing.
Related MCP server: Limetest MCP Server
Install
Works with Claude Desktop, Claude Code, Cursor, Windsurf, Codex, Gemini CLI, Freebuff and every other MCP client — pick whichever route fits:
Route | How |
npm (canonical, fastest) |
|
MCP Registry (registry-aware clients discover it automatically) |
|
Any client, no npm account needed |
|
Claude Desktop, zero Node setup | |
Remote-only clients (ChatGPT connectors) |
|
One line for the common clients:
claude mcp add playwright-e2e -- npx -y playwright-e2e-mcp
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","playwright-e2e-mcp"]}'Details and per-client config: Installation · MCP client configuration.
Tools
Tool | Purpose |
| Run Playwright tests and return stats, failures, diagnostics and hints; reports progress, and |
| Progress or final result of a background run; can wait for it or cancel it |
| Deep analysis of one failure: stack, expected/actual, DOM snapshot at failure (from the Playwright trace), failed requests, console errors, next steps |
| Patch a failure: a drifted locator is replaced with one proven unique on the DOM at failure, changed copy gets the new expected text; |
| Open a URL headlessly (optionally signed in, after steps) and return elements with verified Playwright locators, CSS selectors, visibility, text and console output |
| List available tests ( |
| Check a CSS selector, Playwright selector or locator expression against a live page: validity, match count, and a sturdier verified locator for each match |
| Scaffold a Playwright test from a description using the project's real selectors and verified locators from the live page |
| Visual regression: screenshot before/after a change and report what moved and how colors shifted |
| Run a failing test 2–10 times with retries disabled and return an evidence verdict: |
| Rank flaky, regressed and broken tests from the local run history, group failures by cause, or show one test's timeline |
Two prompts package the main workflows for clients that show them as slash commands:
fix-failing-test (optional test) and triage-flaky-tests.
Page state: signed-in pages, headers and steps
inspect-page, validate-selector and compare-visual-state accept the same page-state
arguments (suggest-fix accepts storageState and headers for a live url):
Argument | Type | Description |
| string | Path inside the project to a Playwright storageState JSON (cookies + localStorage), e.g. |
| object | Extra HTTP headers for every request, e.g. |
| array | Up to 20 steps run after load: |
| object |
|
A failing step reports which one failed and how many ran. When the page lands on a login
screen, the response says so and suggests storageState.
run-test
Argument | Type | Description |
| string | Project directory inside the configured root (default: server working directory) |
| string[] | Files/directories relative to the root; |
| string | Only run tests whose title matches this regex |
|
| Playwright project to run (matched against config project names) |
| boolean | Visible browser window |
| number | Hard wall-clock limit for the run (default |
| number | Per-test timeout passed to Playwright |
| number | Passed through to Playwright |
| string |
|
| boolean | Auto-retry failures once before reporting them (default |
| boolean | Only re-run tests that failed in the previous run (Playwright |
| string[] | Extra Playwright flags from an allowlist ( |
| boolean | Return a run id at once; poll |
Progress: clients that send a progress token get a notification as each test finishes
(3/12 tests done, 1 failed). Every run is appended to the local history used by
analyze-history.
Flakiness handling: by default the server injects --retries=1 (unless the config
already sets retries), so a test that passes on the retry is reported as flaky,
not failed. Traces are captured automatically (--trace=retain-on-failure) so
get-failure can show the DOM at the moment of failure.
Example result:
## Playwright run — ❌ FAILED
**Command:** `playwright test --config playwright.config.ts tests/checkout.spec.ts --reporter=json`
**duration 4.2s · exit 1 · config `playwright.config.ts`**
| passed | failed | flaky | skipped | duration |
| ---: | ---: | ---: | ---: | ---: |
| 0 | 1 | 0 | 0 | 1.1s |
### ❌ 1 failing test(s)
### 1 of 1. checkout.spec.ts › pays with card
**File:** `checkout.spec.ts:5` | **failed · server-unreachable**
### ⚠️ SERVER_NOT_RUNNING
Your app (dev server) does not appear to be reachable. Start it in another terminal
(e.g. npm run dev / npm start), keep it running, then retry — or configure `webServer`
in playwright.config.* so Playwright starts it automatically.get-failure
Argument | Type | Description |
| number | 1-based failure index from the last run (default |
| string | Only used when re-reading the stored report |
Returns the message/code frame, expected vs actual, stack, failure kind with a
diagnosis, the test's console output, the DOM snapshot from the Playwright trace
(plus the failed action, its selector, and the action log leading up to it),
the network requests that failed (4xx/5xx, dead endpoints, no-response — with
method, URL, status and resource type), the console errors/warnings the page
logged before the failure, and
numbered next steps (re-run this single test by file:line, headed/debug mode,
validate-selector when the message mentions a locator, …).
inspect-page
Argument | Type | Description |
| string | Full http(s) URL to open (required) |
|
|
|
| string | Project whose Playwright launches the browser |
| string | Inspect matches of this selector instead of the whole DOM |
| string | Wait for a selector (CSS or |
|
| Navigation wait condition |
| boolean | Include the rendered HTML (capped) |
| number | HTML cap, default |
| number | Overall limit, default |
| — |
Each element comes with Playwright locators that were checked on the page to resolve to exactly that element, best first (role + accessible name, test id, label, placeholder, text, then CSS), plus its unique CSS selector, visibility, box and text. Console messages are captured, errors first. A locator map looks like:
1. `getByRole('button', { name: 'Proceed to checkout' })` — button "Proceed to checkout" · also `getByTestId('checkout-btn')`
2. `getByLabel('Search')` — searchbox "Search"list-tests
Argument | Type | Description |
| string | Project directory |
| string | Config path or 1-based index |
| string | Restrict scanning to a directory (must stay inside the project) |
| string | Case-insensitive substring filter on |
| number | Max tests returned, default |
Uses playwright test --list when Playwright works, and falls back to a source scan
(keeping the reason) when the install or a spec file is broken.
validate-selector
Argument | Type | Description |
| string | Live page to test against (required) |
| string | CSS, a Playwright selector ( |
| string | Project whose Playwright launches the browser |
| number | Overall limit, default |
| — |
Locator expressions are parsed into a whitelisted call chain (getByRole, getByText,
getByLabel, getByPlaceholder, getByAltText, getByTitle, getByTestId, locator,
first, last, nth, filter) and are never evaluated as code.
Verdicts: ✅ VALID — N matches with a sample of matches and a unique locator for each,
a strict-mode warning when more than one element matches, a hint when a CSS selector is
brittle (:nth-child, long descendant chains, XPath), ✅ VALID — 0 matches with debugging
advice, or ❌ INVALID with the parse error.
generate-e2e-test
Argument | Type | Description |
| string | What the test should cover (required) |
| string | Page the test starts on (default: |
| string | Where to write the spec (default: detected |
| boolean | Write the file to disk (default |
| boolean | Replace an existing spec at the target path; only specs this tool generated can be replaced |
| boolean | Cross-check selectors against the live page (default on when a URL is known) |
| string | As with the other tools |
Reads the agent's recent changes (git status, falling back to git diff HEAD~1,
then recent mtimes), extracts the locators those files actually declare
(data-testid, getByRole, aria-label, placeholder, id, name, element text),
ranks verified-live selectors first, writes a spec built from them, and reports each
selector with its source file:line. When the source has no usable selectors, it uses the
verified locators from the live page that best match the description.
compare-visual-state
Argument | Type | Description |
| string | Page to capture (required) |
| string | Baseline id, e.g. |
|
|
|
| string | Capture just this element |
| boolean | Capture the full scrollable page |
| number | Percent of pixels that may differ (default |
| number | Per-pixel channel delta considered different (default |
| — | As with |
| — |
The first call saves a baseline under .pw-mcp/visual/ (add that to .gitignore, or
commit it for CI comparisons). Later calls report changed-pixel counts, merged
regions ((x, y) 120×40 — 1,200 px), the average color shift ("blue → red"),
and write a red-highlighted diff image for review.
diagnose-flaky
Argument | Type | Description |
| string[] | Tests to diagnose ( |
| number | Times to run them, |
| — | As with |
| number | Hard wall-clock limit per run (default |
| string | Project directory |
Every run executes with --retries=0 and auto-retry disabled, so each result is
honest evidence. The response contains a per-run table (status, duration, first
failure), the count of distinct normalized error signatures, and one of:
❌ CONSISTENTLY FAILING — failed every run (same error → reproducible bug, different errors → still broken, just noisy). Fix it; it is not flaky.
⚠️ FLAKY — some runs passed. Includes
N of Mcounts and whether the failures share one signature (real intermittent bug) or vary (timing/environment instability).✅ NOT REPRODUCING — passed every re-run; the original failure was one-off.
The last run is stored, so get-failure can analyze it immediately afterwards.
suggest-fix
Argument | Type | Description |
| number | 1-based failure index from the last run (default |
| boolean | Write the fix into the spec (default |
| boolean | After applying, re-run the test and revert the file if it still fails (default |
| boolean | Allow applying a changed expected text/value (default |
| string | Heal against this live page instead of the DOM snapshot in the failure trace |
| — | For |
| — | As with |
Two kinds of fix:
Locator drift. The broken locator is read from the error, its words are matched (with synonyms such as btn → button, checkout → proceed to checkout) against every element in the DOM at failure, and the best match's verified-unique locator replaces it. Confidence is
highwhen the match is clear,mediumotherwise.Changed copy. For
toHaveText,toContainText,toHaveValue,toHaveTitle,toHaveURLandtoHaveAttribute, it proposes the value the page actually showed. It is only applied withallowExpectationUpdate: true.
The response is a unified diff. With apply: true the file is written only if it has not
changed since the failure, the single test is re-run (same project, no retries), and the
edit is reverted when it is still red. If nothing on the page shares the locator's
distinctive words, it refuses and reports that the element is most likely missing.
get-run-status
Argument | Type | Description |
| string | Run id from |
| number | Wait up to this long (max 55) for the run to finish |
| boolean | Stop the run and kill its browsers |
While running it reports tests done / total and failures so far; once finished it returns
the full run-test result, which also becomes the last run for get-failure and
suggest-fix. The 20 most recent runs are kept in memory.
analyze-history
Argument | Type | Description |
|
|
|
| string | Substring of a test's file or title (filters every view) |
| number | How many recent runs to analyze (default |
| number | Max rows (default |
Every run-test, diagnose-flaky and suggest-fix run appends a line to
<project>/.playwright-e2e-mcp/history.jsonl (the folder carries its own .gitignore;
nothing leaves the machine; PW_MCP_HISTORY=0 turns it off). The flaky view separates
tests that flip between pass and fail, tests that started failing and stayed red
(regressed), and tests that failed every run (broken), each with a PFFP~ timeline.
Installation
Requirements:
Node.js ≥ 20 (the server is built on MCP SDK v2 — the
2026-07-28spec line)A project with
@playwright/testinstalled and browsers available (npx playwright install chromium)
No npm account needed — install straight from GitHub (the prepare script
builds dist/ automatically on install). Pin a release tag: an unpinned
github:trajectiq-ai/E2E runs whatever is on the default branch at that moment.
npx -y github:trajectiq-ai/E2E#v0.2.2
npm install -D github:trajectiq-ai/E2E#v0.2.2 @playwright/test # or as a project dependency
npx playwright install chromiumOr grab the packaged tarball from the repo's GitHub Releases page and install it locally:
npm install -D https://github.com/trajectiq-ai/E2E/releases/download/v0.2.2/playwright-e2e-mcp-0.2.2.tgzListed in the official MCP Registry as
io.github.trajectiq-ai/E2E — registry-aware clients discover it there, and every
v* release tag republishes the entry from CI via server.json.
MCP client configuration
Claude Code / generic (project-scoped):
{
"mcpServers": {
"playwright-e2e": {
"command": "npx",
"args": ["-y", "github:trajectiq-ai/E2E#v0.2.2"],
"env": { "PW_MCP_PROJECT_ROOT": "/absolute/path/to/your/project" }
}
}
}Claude Desktop / Cursor / Windsurf: add the same block to their MCP config file.
The server uses its working directory as the project root; set PW_MCP_PROJECT_ROOT
when the client launches it somewhere else (e.g. your home directory).
Codex / VS Code / Copilot CLIs:
codex mcp add playwright-e2e -- npx -y github:trajectiq-ai/E2E#v0.2.2
code --add-mcp '{"name":"playwright-e2e","command":"npx","args":["-y","github:trajectiq-ai/E2E#v0.2.2"]}'Codex's defaults fight this server: the first launch clones the repo and runs
tsc (measured 30 s on a cold npx cache, against a 10 s
startup_timeout_sec default), and a Playwright run with retries beats the 60 s
tool_timeout_sec default. Raise both in ~/.codex/config.toml:
[mcp_servers.playwright-e2e]
command = "npx"
args = ["-y", "github:trajectiq-ai/E2E#v0.2.2"]
startup_timeout_sec = 60
tool_timeout_sec = 600Claude Desktop (one-click): download and double-click the .mcpb Desktop
Extension attached to the latest release —
the bundle ships its own dependencies, so no Node setup is required. On install it
asks you to pick your project root (required, no default: choose the project
folder, not your home directory) and wires it
into PW_MCP_PROJECT_ROOT, so the tools point at a real project from the first call.
Claude Code:
claude mcp add playwright-e2e -- npx -y github:trajectiq-ai/E2E#v0.2.2Gemini CLI / Qwen Code: paste the mcpServers block above into
.gemini/settings.json (Qwen Code: .qwen/settings.json) — both speak the same
MCP settings format.
Freebuff / Codebuff (project-scoped): this repo ships a committed
.agents/mcp.json, so opening the checkout in Freebuff
attaches the server workspace-wide — no global config needed. Your own projects
can do the same: drop an mcp.json with the block above into their .agents/
directory. Freebuff asks you to trust a repository's .agents/ on first run.
All tools ship MCP tool annotations (readOnlyHint, destructiveHint,
idempotentHint, openWorldHint), so clients can show accurate safety prompts
before running anything.
From a local checkout:
{
"mcpServers": {
"playwright-e2e": {
"command": "node",
"args": ["/path/to/playwright-e2e-mcp/dist/index.js"],
"env": { "PW_MCP_PROJECT_ROOT": "/path/to/your/project" }
}
}
}Hosted endpoint (ChatGPT & remote clients)
Some clients — ChatGPT custom connectors especially — only accept remote
HTTPS MCP servers and refuse to spawn a local npx process. This repo ships a
Streamable HTTP bridge for exactly that case:
Endpoint |
|
Transport | MCP Streamable HTTP ( |
Auth | bearer token ( |
Source |
The bridge runs the same createServer() as the stdio transport; the SDK
serves every request with a fresh server instance, which is what a serverless
function wants. test/http-bridge.test.mjs drives the real Node adapter over
node:http so a broken bridge fails in CI, not in ChatGPT.
This deployment is token-protected. https://playwright-e2e-mcp.vercel.app
has PW_MCP_HTTP_TOKEN set, so a request without Authorization: Bearer <token>
— including a ChatGPT connector created with authentication None — gets
401 {"error":"unauthorized"}. Ask the operator for the token, or run your own
bridge (below).
Open vs. token-protected. With PW_MCP_HTTP_TOKEN set (at least 16
characters; use a random value) every tool is served to clients that send
Authorization: Bearer <token>, and every other request gets 401. Without that
variable anyone who reaches the URL can call the bridge, so it degrades to
list-tests and get-failure in restricted mode: list-tests scans sources
instead of running playwright test --list (which would execute the project's
config), callers cannot pick another projectRoot, and nothing spawns a process,
drives a browser or writes a file. Child processes started over HTTP get only an
allowlisted environment. PW_MCP_ALLOWED_HOSTS (comma separated, * for any)
limits the accepted Host header; without a token and without that variable,
only localhost names and the deployment's own Vercel hostnames are accepted
(DNS-rebinding protection).
Add it to ChatGPT: Settings → Connectors → turn on Advanced → Developer
mode → Create custom connector → paste the endpoint above. Authentication
None only works against a bridge with no token (read-only tools), so for this
deployment choose the connector's API key authentication and send
Authorization: Bearer <token> — or point the connector at your own deployment
with PW_MCP_HTTP_TOKEN unset.
To self-host the open, read-only variant, deploy this repo with
PW_MCP_HTTP_TOKEN unset; nothing else changes.
Codex can also take the remote transport instead of spawning npx, if you'd
rather not ship Playwright to every machine. Point it at the token through an
environment variable so the secret stays out of the config file:
export PW_MCP_HTTP_TOKEN=… # ask the operator for the value
codex mcp add playwright-e2e-remote --url https://playwright-e2e-mcp.vercel.app/api/mcp \
--bearer-token-env-var PW_MCP_HTTP_TOKENWhat to expect: list-tests works and reports the specs bundled with the
deployment. Even with a token, tools that spawn a browser (run-test,
inspect-page, validate-selector, diagnose-flaky, …) cannot download
Chromium in a serverless function, so they return their normal NO_PLAYWRIGHT
hint. Use the stdio install for real runs; the hosted endpoint is for discovery
and for clients that cannot run local processes.
# verify the handshake without any client (the live deployment needs the token;
# drop the authorization header and it answers 401 instead)
curl -X POST https://playwright-e2e-mcp.vercel.app/api/mcp \
-H 'content-type: application/json' \
-H 'accept: application/json, text/event-stream' \
-H "authorization: Bearer $PW_MCP_HTTP_TOKEN" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1.0"}}}'Redeploy after a change: merge to main. Vercel's Git integration deploys
every push, so there is no token to manage and no CLI step — watch the
Vercel commit status for the deployment result.
Configuration
Environment variable | Default | Purpose |
| server cwd | Default project root for every tool |
| — | Default |
| on |
|
| — | Extra directories a caller may pass as |
| — | HTTP bridge only: bearer token (16+ characters) that unlocks all tools (see above) |
| localhost + Vercel hostnames when there is no token | HTTP bridge only: comma-separated |
| — | HTTP bridge only: comma-separated extra variables passed to test runs (e.g. |
|
| HTTP bridge only: how many test runs and browser probes may run at once |
| off (on for the HTTP bridge) |
|
|
|
|
|
|
|
Logs always go to stderr — stdout is reserved for the MCP protocol. Unrecognized values
fall back to the default, so a typo in LOG_LEVEL logs at info and one in LOG_FORMAT
logs as text rather than silencing the server. With LOG_FORMAT=json each line is a single
object (time, level, message plus context):
{"time":"2026-10-07T08:00:00.000Z","level":"info","message":"server ready","name":"playwright-e2e-mcp"}Typical workflow
run-test{ "testFiles": ["tests/checkout.spec.ts"] }, or{ "background": true }for a long suite andget-run-status{ "waitSeconds": 30 }until it finishes.get-failure{ "index": 1 }to read the code frame, expected/actual, the DOM snapshot at failure, the failed network requests and the page's console errors.suggest-fix{ "index": 1 }for a locator or copy change: review the diff, then{ "index": 1, "apply": true }to write it and verify with a re-run. If it reports the element is missing, fix the app, not the test.For anything it cannot patch:
inspect-page{ "url": "…", "view": "locators" }(addstorageStateoractionsto reach the right screen), thenvalidate-selectorto prove the new locator before editing.If a failure looks intermittent:
diagnose-flaky{ "runs": 3 }for fresh evidence, andanalyze-historyto see how the test behaved across earlier runs.After changing CSS or components:
compare-visual-state{ "url": "…", "name": "checkout" }.Re-run only what failed with
run-test{ "lastFailed": true }until green.
To write a new test, start with generate-e2e-test { "description": "checkout with a saved card" }.
Benchmark
bench/token-benchmark.mjs breaks a checkout button's test id
on a storefront page and fixes it two ways: with this server (run-test → suggest-fix →
apply) and with Playwright MCP plus a shell (playwright test, browser_navigate +
browser_snapshot, edit, re-run). Latest result (full output):
Step | playwright-e2e-mcp | Playwright MCP + shell |
Tool definitions (sent with every request) | 4,565 | 4,726 |
Run the failing test | 246 | 325 |
Find the element / propose the fix | 260 | 2,286 |
Apply + verify (re-run) | 257 | 34 |
Total for one fix | 5,328 | 7,371 |
Tokens are characters ÷ 4. The Playwright MCP route is a lower bound: it assumes the agent picks the right locator from the snapshot on the first try, which the server does for it here.
Edge cases handled
Situation | Behaviour |
No Playwright installed |
|
Dev server not running | Failure classified |
Test exceeds | Process group is killed (SIGINT→SIGKILL on POSIX, |
Browser crashes | Classified |
Windows backslashes | All paths normalized lexically ( |
Flaky tests | Failing tests are automatically retried once ( |
Trace/DOM context |
|
Slow re-runs after a fix |
|
Several | Returns a numbered menu ( |
Syntax error in a spec |
|
MCP client disconnects | Per-request |
Disk full |
|
Malicious paths |
|
Security notes
No shell: Playwright is spawned as
node <playwright/cli.js> …with an argument array — no command interpolation. Extraargsare limited to an allowlist of Playwright flags, each with a checked value (--flag=value), so a caller cannot swap in another--configor--outputor hand git an option through--only-changed. Paths with a segment starting with-are rejected, so a test file name cannot be read as a flag.Path sandbox: user paths must stay inside the project root, checked lexically and again with symlinks resolved. A caller-supplied
projectRootmust sit insidePW_MCP_PROJECT_ROOT(or aPW_MCP_ALLOWED_ROOTSentry).Generated code:
generate-e2e-testonly writes*.spec.*/*.test.*files, only overwrites specs whose header it wrote itself, never writes through a symlink, and escapes every value it puts into strings or comments.HTTP bridge: read-only tools unless
PW_MCP_HTTP_TOKENis set; child processes started over HTTP get only an allowlisted environment (PATH,HOME, temp dirs, locale,CI,PLAYWRIGHT_*,npm_config_*without embedded credentials, plus anything inPW_MCP_PASSTHROUGH_ENV), at mostPW_MCP_MAX_CHILDRENrun at once (a disconnected client frees its slot immediately), and internal error messages are not returned to clients. The allowlist only covers the child's own environment: test code runs as the same OS user, so on Linux it could still read the server's startup environment from/proc. That is why only token holders can run project code; keep other secrets out of the bridge's environment, or run it under a separate user.SSRF guard: on the HTTP bridge (or with
PW_MCP_BLOCK_PRIVATE_URLS=1) the URL tools refuse hosts that resolve to loopback, private, link-local/metadata or reserved addresses, and route the browser through a local proxy that applies the same check to every redirect hop and subresource; IPv6 forms that embed an IPv4 address (mapped, NAT64, 6to4, Teredo) are blocked too, and WebRTC UDP is disabled so a page cannot reach the network around the proxy. It is off for stdio by default because openinghttp://localhostdev servers is what these tools are for.Cleanup: temp report/script files are written to the OS temp dir and removed; child processes are tracked and killed on shutdown. Trace decompression has one size budget per archive, and PNGs are capped at 16,384 px per side and 50 M pixels.
Development
src/
├── index.ts # bin entry point (--version/--help, main-module guard)
├── server.ts # McpServer setup, tool registration, shutdown handling
├── tools/ # the eleven tools, the probe script + shared plumbing
├── prompts.ts # fix-failing-test, triage-flaky-tests
├── utils/ # playwright-runner, report-parser, project-detector, path-utils,
│ # logger, trace-reader (trace.zip → DOM/network/console),
│ # image-diff (PNG codec + pixel diff), change-analyzer,
│ # locator-expr, run-history, run-registry
└── types/ # shared interfaces and the ErrorKind taxonomyBuilt on @modelcontextprotocol/server v2 (the 2026-07-28 MCP spec line) with
Zod v4 standard schemas; every tool declares spec tool annotations.
npm install
npm run build # tsc → dist/ (zero errors)
npm test # build + test/run-tests.mjs (unit tests, any Node ≥20)
npm run e2e # build + e2e/run.mjs: live MCP ↔ Playwright integration suite
npm run mcpb # build + bundle dist/ into playwright-e2e-mcp-<version>.mcpb (+ .sha256)
npm run mcpb:smoke # ...then install that bundle in a temp dir, launch it and assert the handshake
npm run mcpb:sign # ...then sign it with MCPB_CERT/MCPB_KEY → <version>.signed.mcpb
npm run skill:zip # just the archive: dist-skill/playwright-e2e-mcp.zip
npm run skill:package # the uploadable folder: ../agensi-skill-<version>/ (skill copy, README, zip)
npm run skill:check # build, then fail if a packaged copy has drifted from skill/Tests cover the report parser (sample Playwright JSON, trace attachments), path utils
(Windows and macOS paths, sandboxing), the project detector (temp-dir fixtures: config
discovery, multiple configs, missing install, test-file scanning), the shared tool
helpers, the trace reader (synthetic trace.zip: error, failed action, DOM snapshot,
*.network failed-request parsing, console error/warning events), the image diff
(PNG round-trip, regions, color shift, dimension changes), the change analyzer
(selector extraction, git + mtime paths), the flaky verdict logic
(failure signatures, CONSISTENTLY FAILING / FLAKY / NOT REPRODUCING / NO TESTS RAN),
the HTTP bridge (token, restricted mode, Host allowlist), the security
regressions (sandbox escapes, argument smuggling, symlink writes, code injection,
SSRF ranges, env scrubbing, decode limits), the .mcpb manifest, the .mcpb zip
extractor, the skill package (byte-identical rebuilds, a CRLF checkout hashing the
same, refusal of junk/binary/symlinks, and drift detection between skill/ and a
packaged copy), and the MCP config files.
Integration suite (npm run e2e)
Unit tests prove the logic; the integration suite proves the loop. It boots the real
server over stdio against a live fixture app and a Playwright project under
e2e/fixture/, then drives it exactly like an MCP client and asserts ~40 behaviours
that only appear end-to-end:
initialize handshake, 11 tools, 2 prompts, spec tool annotations and object input schemas,
verified locators, locator validation, signed-in pages (
storageState) and page steps,background runs with progress, run history views, and
suggest-fixhealing a drifted locator and verifying it with a re-run (and refusing when the element is missing),live DOM inspection, CSS selector validation (matches, zero matches, engine syntax, parse errors), dead-server detection,
visual regression: baseline → unchanged compare →
blue → reddiff detection,run-testpass/fail/lastFailedstats and meta lines,get-failuretrace diagnostics: DOM at failure, the 404 network request, theconsole.errormessage, diagnosis and next steps,auto-retry turning a first-run failure into
PASSED (1 flaky),diagnose-flakyverdict FLAKY (2 of 3 runs) with retries disabled,generate-e2e-testwriting its scaffold, plus error paths (missing test path, unknown tool, unreachable server).
First run needs the browser once: npx playwright install chromium.
CI runs the suite on Ubuntu and Windows (see .github/workflows/ci.yml).
Signing the bundle
MCPB_CERT=~/signing/cert.pem MCPB_KEY=~/signing/key.pem npm run mcpb:signSigning appends a PKCS#7 signature block to the bundle, so the signed file is
the reproducible bundle plus that block. That ordering is the point: the bytes
CI proved identical across platforms stay inside the signed file, and
mcpb unsign returns them exactly. scripts/sign-mcpb.mjs asserts that round
trip on every run and deletes its output if it fails, which is what keeps
<version>.signed.mcpb tied to <version>.mcpb.
The private key is read from a path you pass in (--cert/--key or
MCPB_CERT/MCPB_KEY) and never from the repository — keep it outside version
control, and back it up: the same key across releases is what makes the signer
recognisable.
What signing does and does not buy you is decided by the certificate, not by
this script. mcpb verify and mcpb info accept a signature only when the
certificate chains to something in the OS trust store, so with a self-signed
certificate a host still reports Not signed — the signature is real
integrity and key continuity, verifiable by the fingerprint the script prints,
but not third-party identity. A certificate issued by a public CA, or an
internal one installed on your machines, is what makes hosts report it as
signed.
Assembling a release
npm run mcpb && npm pack # the bundle, then the npm tarball
npm run release:assemble -- --tag v0.2.3 # sign, checksum and write the notes
npm run release:publish -- --tag v0.2.3 --assets dist-release/assets \
--title-file dist-release/RELEASE_TITLE.txt \
--body-file dist-release/RELEASE_BODY.md --download dist-release/downloaded
npm run release:verify -- v0.2.3 --dir dist-release/downloaded
npm run release:publish -- --tag v0.2.3 --publishPushing a v* tag runs exactly that sequence in
.github/workflows/release.yml. The release used to be assembled by hand, and
the notes were written from what those commands printed: every hash in them was
a copy, and nothing compared the bundle hash printed there with the bundle that
shipped.
scripts/assemble-release.mjs stages the whole release into
dist-release/assets/: the bundle and its sidecar, the signed bundle and the
signing certificate when a key is configured, the npm tarball, the skill package
built by the same writer as npm run skill:zip, and SHA256SUMS.txt covering
every file it staged. It empties that directory first, so a file left over from
an earlier run cannot be hashed and published as if this run had produced it.
The notes are half written, half generated. release-notes/<tag>.md holds the
prose — its H1 becomes the release title and has to name the tag — and the script
appends the asset table, the reproduction recipe carrying the real bundle hash,
the signer and fingerprint, and the commit list. The prose is held to the same
rule: a 64-character hex string in it has to be a hash this release publishes,
and writing an ### Assets section of your own fails the assembly rather than
shipping a second, staler table. See release-notes/README.md.
scripts/publish-release.mjs fills the release as a draft, uploads the
assets, downloads every one of them back and compares it byte for byte, and only
--publish makes it public — after release:verify has re-derived the release
from the tag. Re-running is safe: a draft this workflow assembled is refreshed in
place, while a release it did not assemble, or one already published, is refused
by name instead of overwritten.
Signing is optional and its absence is stated rather than hidden: without the
MCPB_KEY and MCPB_CERT repository secrets the release is assembled and
published unsigned, with a notice in the run and a line saying so in the
notes. Add both — the private key stays out of the repository and out of the logs:
gh secret set MCPB_KEY --repo trajectiq-ai/E2E < ~/signing/key.pem
gh secret set MCPB_CERT --repo trajectiq-ai/E2E < ~/signing/cert.pemCreating the release and uploading its assets needs contents: write, which the
job requests for itself; if it is refused, the run names the setting to change.
Verifying a published release
git checkout v0.2.2 && npm ci && npm run mcpb
npm run release:verify -- v0.2.2The release notes promise that the tag rebuilds the published bundle byte for
byte. scripts/verify-release.mjs checks that promise from a checkout of the
tag: the published bundle against your fresh build, every checksum the release
publishes (and that nothing shippable is missing from SHA256SUMS.txt), a
signed bundle against the bytes its signature wraps, the skill package
against a rebuild from the same tag, and that the published archive still
extracts with that version's manifest. The skill package is required of a tag
whose tree carries the publishing workflow; older tags predate it, so there it
is checked when present and skipped when not, like the signing certificate.
.github/workflows/verify-release.yml runs it on demand for any past tag, and by
itself for a release published by hand. A release assembled by
.github/workflows/release.yml runs the same check on its own assets before
it is published — GitHub does not raise release: published for a release created
with the repository's own token, so the check is a step there rather than a
trigger. The .tgz is checked for download integrity only: npm
records the packer's line endings, so a tarball packed locally on another OS is
legitimately different.
Packaging the agent skill
npm run skill:package # ../agensi-skill-<version>/, ready to upload
npm run skill:zip # only the archive: dist-skill/playwright-e2e-mcp.zip
npm run skill:check # build, then compare every copy
npm run skill:check -- ../agensi-skill-0.2.2skill/ is an Agent Skill — the
instructions an agent follows to drive this server — and the marketplace gets
the same Markdown as a zip with playwright-e2e-mcp/ at its root.
scripts/package-skill.mjs produces that zip, so the uploaded copy is built
rather than assembled by hand, and it refuses what the marketplace rejects: a
missing SKILL.md, a non-text file, a symlink, Thumbs.db.
skill:package writes the whole folder a release would otherwise assemble by
hand — the skill copy, skill/README.md and the archive — to
../agensi-skill-<version> (or --out <dir>), then verifies it with the same
comparison check uses, so a successful exit means the folder really is the
skill tree. It is idempotent: a folder that already matches is left untouched.
A folder that has drifted is refused instead of overwritten, because it may
already be published — bump version for a new package, or pass --force to
replace that one.
The archive comes from the same deterministic writer as the .mcpb, so the
same tree always hashes the same, and text payloads are LF-normalised first:
a CRLF checkout builds the bytes an LF one does. CI runs skill:check, generates
a package folder, and uploads the built zip as an artifact
(.github/workflows/ci.yml).
Every release publishes the package too. Assembling a release builds it from
the release tag and attaches it as <name>-<version>-skill.zip, and the
.github/workflows/release.yml run downloads that asset back off the draft and
byte-compares it with the build before anything is published. The marketplace
upload is that asset, not a folder someone assembled — a deterministic archive
means "the same build as the tagged release" is an exact statement, and
release:verify checks it again against a rebuild from the tag.
.github/workflows/publish-skill-package.yml stays for the repair path: run it
by hand to attach or rebuild the package for a release that is missing it.
skill:check is the other half of that, and the part that matters when a
package has already been uploaded. It builds twice (the bytes must agree),
reads the archive back and compares its contents with skill/ file by file,
then compares a packaged copy — ../agensi-skill-<version> by default, or a
path you pass: the package folder, the bare playwright-e2e-mcp/ folder, or
the zip on its own. Comparisons are by content, never by archive bytes, because
a zip written by a platform zip tool carries real timestamps that say nothing
about whether the Markdown inside it is current. Any file that differs, is
missing or is extra is named on stderr and the command exits 1 — the repository
copy and the uploaded copy are the same document, so an edit to either one
should not be invisible.
Try the example
With Playwright installed in your project:
npx playwright test examples/sample-test.spec.tsor ask your agent to call run-test with
"testFiles": ["examples/sample-test.spec.ts"] — it hits the public
example.com page, so it verifies browsers, network and the MCP
pipeline in one shot.
License
MIT
Available Tools
11 toolsanalyze-historyAnalyze run historyARead-onlyIdempotentInspect
Analyze the local run history (.playwright-e2e-mcp/history.jsonl, written by every run): 'flaky' ranks tests that flip pass/fail and lists broken or regressed ones; 'patterns' groups failures by cause; 'test' shows one test's timeline.
| Name | Required | Description | Default |
|---|---|---|---|
| test | No | Filter: substring of file or title | |
| view | No | Default 'flaky' | |
| limit | No | ||
| lastRuns | No | Runs to analyze (default 30) | |
| projectRoot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and openWorldHint=false, so the safety profile is covered. The description adds meaningful context beyond that: the data source is a local JSONL file written by every run, and it discloses what the flaky view surfaces (pass/fail flips, broken or regressed tests). It omits defaults/limits and any note on history retention or staleness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence with a parenthetical, front-loading the action and data source before enumerating view behaviors. Every clause carries information; the view enumeration is the longest but earns its place. Slightly heavy in a single sentence, but no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must carry more weight, and it does explain the return semantics of each view. However, projectRoot (presumably how the history file is located) and limit/lastRuns bounds are left entirely to the schema, leaving the picture partially incomplete for a 5-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 60%, so baseline is 3. The description meaningfully expands the 'view' enum by explaining the semantics of each value, adding value over the schema's bare listing. But it says nothing about limit, projectRoot, or lastRuns, so the gap in coverage is not compensated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Analyze) and resource (local run history), and names the exact backing file .playwright-e2e-mcp/history.jsonl. It enumerates the three view modes with their behaviors, so the agent knows what output families are available. It does not explicitly differentiate from siblings like diagnose-flaky or get-failure, which keeps it from a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells the agent what each 'flaky', 'patterns' and 'test' view does, which implies which view to pick for a given need. However, it never states when to use this tool instead of diagnose-flaky, get-failure, or get-run-status, nor any prerequisites. Usage is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare-visual-stateCompare visual stateADestructiveIdempotentInspect
Visual regression check: screenshot a live page (or element), compare it to a stored baseline, and report how many pixels changed, WHERE they changed (bounding regions), and the average color shift ("blue → red"). First call saves the baseline; later calls compare. Writes a red-highlighted diff image for review.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Full URL of the page (or element target) to capture | |
| name | Yes | Baseline id, e.g. "checkout-page" or "submit-button" | |
| action | No | 'baseline' overwrites the stored baseline; 'compare' (default) compares against it | |
| actions | No | Steps run after load to reach a state (modal, step 3) | |
| headers | No | Extra HTTP headers, e.g. Authorization | |
| waitFor | No | Wait for this selector before capturing | |
| fullPage | No | Capture the full scrollable page (default false) | |
| selector | No | Capture only this element (selector or locator) | |
| viewport | No | ||
| timeoutMs | No | ||
| tolerance | No | Percent of pixels that may differ before reporting a change (default 0.1) | |
| waitUntil | No | ||
| projectRoot | No | Project directory; defaults to the server working directory | |
| storageState | No | Logged-in session: storageState JSON inside the project, e.g. playwright/.auth/user.json | |
| pixelThreshold | No | Combined per-pixel channel delta considered different (default 60) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=true and openWorldHint=true; the description supports this by disclosing the baseline-save behavior and that a red-highlighted diff image file is written as a side effect, which is real context beyond the flags. It does not warn that re-saving a baseline overwrites the prior one, nor mention auth/session needs (storageState), leaving one meaningful destructive consequence implicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the tool's identity, then the baseline/compare lifecycle, then the artifact written — three useful sentences with no filler. The long parentheticals ('WHERE they changed (bounding regions), and the average color shift ("blue → red")') are arguably denser than needed but still informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 15-parameter tool with nested objects and no output schema, the description usefully summarizes the return content (change count, bounding regions, color shift), substituting for the missing output schema. It still omits where the diff image is written and whether results come back as structured data, so it is strong but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 80%, so the schema largely documents url, name, action, selector, tolerance, and pixelThreshold itself. The description only loosely maps to parameters ('page (or element)' for url/selector, baseline lifecycle for action) and adds no format, default, or unit detail beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource label ('Visual regression check') and then concretely describes the pipeline: screenshot a live page or element, compare to a stored baseline, report pixel deltas, bounding regions, and color shift. An agent can distinguish this from inspect-page, validate-selector, or run-test without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly states the operating model: 'First call saves the baseline; later calls compare,' which tells the agent how to sequence calls. It does not name sibling alternatives or when NOT to use it (e.g., prefer inspect-page for DOM debugging), so it stops short of full routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
diagnose-flakyDiagnose flaky testADestructiveIdempotentInspect
Run a failing test multiple times with retries disabled and return an evidence-based verdict: CONSISTENTLY FAILING (real bug), FLAKY (intermittent — includes pass/fail counts and error-signature variance) or NOT REPRODUCING (one-off). Defaults to the tests that failed in the most recent run.
| Name | Required | Description | Default |
|---|---|---|---|
| runs | No | How many times to run the tests (default 3) | |
| config | No | playwright.config path or 1-based index when the project has several configs | |
| headed | No | Run with a visible browser window | |
| browser | No | Playwright project name to run | |
| workers | No | ||
| testFiles | No | Tests to diagnose ("file:line" supported). Defaults to the tests that failed in the most recent run. | |
| timeoutMs | No | Hard wall-clock limit per run (default 120000) | |
| projectRoot | No | Project directory; defaults to the server working directory |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds behavior the annotations don't convey: retries are disabled during diagnosis, tests are executed multiple times, and the result varies in the described way (pass/fail counts, error-signature variance). Annotations cover safety (destructive/idempotent/openWorld), so the description is free to focus on run semantics, which it does well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence, front-loaded with the action and the three possible verdicts. Every clause carries information; nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter tool with no output schema, the description covers the output shape (verdict categories and what each contains) and the default targeting, which is the critical missing piece. Remaining parameters are handled by the schema, so the gap is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 88% and already documents runs, testFiles, timeoutMs, and config. The description repeats the testFiles and runs defaults without adding format or interaction details, so it does not materially extend the schema. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (run failing test repeatedly), the key constraint (retries disabled), and the output (evidence-based verdict with three named outcomes). An agent can distinguish it from run-test purely from this description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clear context: use when a test is failing and you need to know whether it's a real bug, flaky, or a one-off, and it defaults to the most recent failure set. It doesn't explicitly name a sibling to use instead for plain execution, but the diagnostic framing makes the boundary with run-test obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
generate-e2e-testGenerate an E2E testADestructiveInspect
Scaffold a Playwright test from a description using the project's REAL selectors. Scans the agent's recent file changes (git working tree / last commit, mtime fallback), extracts data-testid / role / label / placeholder / id locators with their source locations, optionally cross-checks them against the live page, writes a ready-to-run spec, and returns next steps.
| Name | Required | Description | Default |
|---|---|---|---|
| file | No | Exact spec path (*.spec.ts / *.test.ts) inside the project instead of the default location | |
| write | No | Write the file to disk (default true) | |
| config | No | playwright.config path or 1-based index | |
| pageUrl | No | Page the test starts on (default: baseURL from playwright.config) | |
| testDir | No | Directory for the generated spec (default: detected testDir or tests/) | |
| overwrite | No | Replace an existing spec at the target path (only specs this tool generated) | |
| description | Yes | What the test should cover, e.g. "login with valid credentials" | |
| liveInspect | No | Verify discovered selectors against the live page (default true when a URL is known) | |
| projectRoot | No | Project directory; defaults to the server working directory |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, openWorldHint=true, and non-idempotent, so the safety profile is covered. The description earns credit for disclosing behavior beyond that: it scans the git working tree/last commit (mtime fallback), extracts specific locator types with source locations, optionally cross-checks the live page, writes the spec, and returns next steps. It only stops short of flagging the overwrite hazard explicitly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that leads with the verb and resource, then enumerates the pipeline steps compactly. Dense but every clause carries information with no filler, so it is easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, open-world generation tool with nine params and no output schema, the description covers the process and notes the return ('returns next steps'). It omits where the spec lands by default and any warning about writing to disk, but is otherwise complete enough to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema documents all nine parameters and the baseline is 3. The description adds contextual meaning for a few (live cross-check implies liveInspect, project scanning implies projectRoot), but does not explain file, write, overwrite, or config beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Scaffold) and resource (Playwright test spec) and adds the key qualifier 'using the project's REAL selectors', so the intent is unambiguous. It does not name or contrast with any sibling (e.g. validate-selector, inspect-page, run-test), so it stops short of explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the mechanism but never says when to reach for this tool versus siblings like validate-selector, inspect-page, or run-test. No when-to-use, when-not-to-use, or ordering guidance is provided, leaving routing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-failureAnalyze a test failureARead-onlyIdempotentInspect
Analyze one failure from the most recent run-test execution: full message and code frame, expected vs actual, stack trace, failed network requests (4xx/5xx and dead endpoints), console errors, failure kind (assertion/timeout/browser-crash/syntax/config/server-unreachable) and concrete next steps. Use run-test first.
| Name | Required | Description | Default |
|---|---|---|---|
| index | No | 1-based failure index from the last run (default 1) | |
| projectRoot | No | Only used to re-read the stored report |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/idempotentHint/openWorldHint=false, so safety is covered; the description adds real value by disclosing that it operates on the stored output of the last run-test execution and by enumerating the failure-kind taxonomy (assertion/timeout/browser-crash/syntax/config/server-unreachable). It omits the failure modes of the tool itself, e.g. what happens when no failures exist or the index is out of range.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence, front-loaded with what is returned, with no filler. It is heavy on enumeration and could be split, but every clause carries information the agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must describe the return value, and it does so thoroughly (message, code frame, expected vs actual, stack trace, network requests, console errors, failure kind, next steps). The remaining gap is edge-case behavior — no failures recorded, invalid index, or stale report — which would matter for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are documented in the schema, including the 1-based index and the default of 1. The description only implies the index scoping ('one failure from the most recent run-test execution') and never mentions projectRoot's re-read role, so it adds little beyond the schema — the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (analyze) and resource (one failure from the most recent run-test execution) plus the exact payload the agent will get back. It is clearly distinguishable from siblings like analyze-history (historical) and suggest-fix (remediation), which serve different intents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use run-test first,' establishing the prerequisite ordering against the sibling that produces the report. It does not, however, say when to prefer suggest-fix or diagnose-flaky after a failure is retrieved, so the routing guidance stops one step short of complete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-run-statusCheck a background runAIdempotentInspect
Progress (tests done/total, failures so far) or the final result of a run-test background run; the result becomes the last run for get-failure. waitSeconds waits; cancel stops it. Default: the latest run.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | No | ||
| cancel | No | ||
| waitSeconds | No | Wait up to this long for it to finish |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare idempotentHint=true, destructiveHint=false and, notably, readOnlyHint=false; the description explains why by disclosing that cancel terminates the run and that waitSeconds blocks. It also states the default-run fallback, which annotations cannot express. It does not restate the schema's 55s wait cap or describe failure/error states, keeping it out of 5 territory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence front-loads the return shape (progress vs final result), then the parameter semantics, then the default. No filler, though the semicolon-chain style is telegraphic and the waitSeconds 55s bound lives only in the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly fills the gap by naming the return payload (tests done/total, failures so far, final result) and the downstream relationship with get-failure. Missing only error/timeout behavior and what a cancelled run returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33% (just waitSeconds), so the description must carry runId and cancel. It does: 'cancel stops it' defines cancel, and 'Default: the latest run' tells the agent runId is an optional selector. It stops short of 5 only because runId's accepted format/scope (e.g., run id vs test id) is left implicit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (retrieve progress or final result of a run-test background run) and names the two siblings it relates to: run-test produces the run, get-failure consumes the result. An agent can place it precisely in the workflow without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies the polling workflow (progress vs final result), explains that waitSeconds blocks and cancel aborts, and states the default (latest run) for the no-arg case. It never explicitly says when NOT to use it versus get-failure or list-tests, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
inspect-pageInspect live page DOMARead-onlyIdempotentInspect
Open a URL headlessly and return its elements with Playwright locators proven unique on the page (role + name first), CSS selectors, visibility, text and console messages. storageState/headers open it signed in; actions click/fill first to reach a state. view 'locators' is the compact map.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Full URL of the page to open (http/https) | |
| view | No | 'locators': compact map of interactive elements (cheapest); 'elements' (default): DOM inventory | |
| actions | No | Steps run after load to reach a state (modal, step 3) | |
| headers | No | Extra HTTP headers, e.g. Authorization | |
| waitFor | No | Wait for this selector first | |
| selector | No | CSS selector: inspect matches instead of the whole DOM | |
| viewport | No | ||
| timeoutMs | No | Overall inspection limit in ms (default 45000) | |
| waitUntil | No | Navigation wait condition (default domcontentloaded) | |
| includeHtml | No | Include the rendered HTML (truncated) | |
| projectRoot | No | Project whose Playwright launches the browser (default: server root) | |
| maxHtmlChars | No | HTML cap when includeHtml is set (default 20000) | |
| storageState | No | Logged-in session: storageState JSON inside the project, e.g. playwright/.auth/user.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already supply readOnlyHint, openWorldHint and idempotentHint, so the safety profile is covered. The description adds meaningful behavior beyond that: the browser runs headless, actions are executed before inspection, and console messages are captured. It doesn't say how failures (nav timeout, missing locator) surface, which is the remaining gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is three tight, front-loaded sentences with no filler; the core behavior leads and the usage hints follow. Phrasing is telegraphic ('storageState/headers open it signed in'), which is efficient but slightly compressed for a 13-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 13 parameters, nested action objects and no output schema, the description does the important work of enumerating what is returned and how auth/actions affect the result. Missing are error/timeout behavior and truncation semantics, but the essentials for correct invocation are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 92%, so the baseline is 3, but the description adds real meaning: storageState/headers are framed as the signed-in path and actions as the way to reach a state (modal, step 3). The view enum is likewise described by cost/size rather than just restating the enum values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb+resource (open a URL headlessly and return its elements) plus the returned payload: locators, CSS selectors, visibility, text, console messages. It is far more specific than a sibling like run-test, but it never explicitly contrasts itself with validate-selector or generate-e2e-test, so the agent must infer the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear operational context: storageState/headers open the page signed in, actions click/fill first to reach a state, and view 'locators' is the compact/cheapest map. It does not name an alternative tool or state when-not to use this one, so it stops short of explicit routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list-testsList available testsARead-onlyIdempotentInspect
List the Playwright tests available in the project (file, line, full title, projects), with optional filtering. Uses playwright test --list when Playwright works and falls back to a source scan when the install or config is broken, reporting why.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max tests to return (default 500) | |
| config | No | playwright.config path or 1-based index when the project has several configs | |
| filter | No | Case-insensitive substring filter on "file › title" | |
| testDir | No | Restrict scanning to this directory (inside the project) | |
| projectRoot | No | Project directory; defaults to the server working directory |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnlyHint, idempotentHint, openWorldHint=false), but the description adds genuinely useful behavior: a primary `playwright test --list` path with a source-scan fallback when the install or config is broken, plus the promise that it reports why. It does not mention cost, latency, or whether the fallback results are less complete, which keeps it from a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: the first front-loads what is listed and what is returned, the second supplies the fallback behavior. No filler, no restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description partially compensates by naming the returned fields and explaining the dual-path execution. Combined with a fully documented 5-parameter schema and zero required parameters, an agent has nearly everything needed; only the shape of the fallback 'reason' report is left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so every parameter (limit, config, filter, testDir, projectRoot) is already documented with defaults and constraints in the schema. The description adds only the generic phrase 'with optional filtering', which maps loosely to the filter param without adding syntax or format detail. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (Playwright tests available in the project) and even enumerates the returned fields (file, line, full title, projects). No sibling tool in the set performs listing, so an agent can distinguish it from run-test, get-run-status, or generate-e2e-test immediately.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The mention of 'optional filtering' implies a discovery use case (finding tests before acting on them), but the description never states when to reach for this tool versus run-test or get-run-status, nor does it set any preconditions. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run-testRun E2E testsADestructiveIdempotentInspect
Run the project's Playwright tests: stats plus each failure's file:line, kind and message. Diagnoses missing installs, dead dev servers, syntax errors, crashes and timeouts (kills the tree, keeps partial results). Reports progress; background: true returns a run id for long suites. Runs are recorded for analyze-history.
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Extra Playwright flags from an allowlist, as --flag or --flag=value (e.g. --repeat-each=3, --shard=1/2, --trace=on) | |
| grep | No | Only run tests whose title matches this regex | |
| config | No | playwright.config path or 1-based index when the project has several configs | |
| headed | No | Run with a visible browser window | |
| browser | No | Playwright project name to run (matched against playwright.config projects) | |
| retries | No | ||
| workers | No | ||
| testFiles | No | Test files/directories relative to the project root; "file:line" is supported. Omit to run the whole suite. | |
| timeoutMs | No | Hard wall-clock limit for the whole run; the process tree is killed past it (default 120000) | |
| background | No | Return a run id at once; poll get-run-status | |
| lastFailed | No | Only re-run tests that failed in the previous run (Playwright --last-failed) — the fast fix → re-run loop | |
| projectRoot | No | Project directory; defaults to the server working directory | |
| testTimeoutMs | No | Per-test timeout passed to Playwright | |
| retryOnFailure | No | Auto retry failing tests once before reporting failure (default true; ignored when retries is set) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by describing the failure taxonomy it diagnoses (missing installs, dead dev servers, syntax errors, crashes, timeouts), the timeout behavior (kills the process tree, keeps partial results), progress reporting, and the background run-id workflow. With destructiveHint=true already declared, it usefully clarifies that a timeout kill is scoped and partial results survive, though it doesn't state side effects like artifact writes or install attempts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four dense sentences, front-loaded with what the tool does and its output before behavioral details. Nearly every clause earns its place, though the telegraphic style ('stats plus each failure's file:line, kind and message') is compact to the point of being terse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter runner with no output schema, the description does the necessary work: it describes the return payload, progress reporting, timeout kill behavior, and how to retrieve results from long suites via background plus get-run-status. Remaining gaps (artifact locations, interaction with sharding) are minor given the rich schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 86% and the schema itself is highly detailed (allowlisted flags, timeout semantics, last-failed loop), so the parameters are already well documented. The description adds the partial-results behavior on timeout and the background polling workflow, but most parameter meaning is carried by the schema rather than the prose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Run the project's Playwright tests') and names the exact output shape: stats plus each failure's file:line, kind and message. It also carves out a distinct role from siblings by framing itself as the runner that records runs for analyze-history and drops a run id for get-run-status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear operational context: background:true returns a run id for long suites and you poll get-run-status, which names a concrete alternative and the condition selecting it. It also points forward to analyze-history. It stops short of explicit when-not-to-use guidance (e.g., when to prefer list-tests or get-failure instead).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
suggest-fixSuggest (and apply) a fixADestructiveInspect
Patch a failure from the last run: a drifted locator is matched against the DOM at failure (trace, or live url) and replaced with one proven unique; changed copy gets the new expected text. Returns a diff and confidence; apply: true writes it, re-runs the test and reverts if still red. Refuses when the element is truly missing (app bug).
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Heal against this live page instead of the trace snapshot | |
| apply | No | Write the fix (default: diff only) | |
| index | No | 1-based failure index (default 1) | |
| config | No | playwright.config path or 1-based index | |
| verify | No | Re-run after applying; revert if still failing (default true) | |
| headers | No | Extra HTTP headers, e.g. Authorization | |
| timeoutMs | No | Verification run limit (default 120000) | |
| projectRoot | No | ||
| storageState | No | Logged-in session: storageState JSON inside the project, e.g. playwright/.auth/user.json | |
| allowExpectationUpdate | No | Allow applying changed expected text (can hide regressions) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations flag destructiveness and non-idempotency, and the description adds behavior annotations cannot express: the verify-then-revert cycle ('re-runs the test and reverts if still red'), the returned artifacts (diff and confidence), and the refusal condition for genuine app bugs. This is exactly the extra context an agent needs before invoking a destructive tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences, front-loaded with the core action and repair mechanism before the apply/verify semantics and the refusal caveat. Every clause carries information, though the middle sentence packs several behaviors into one run-on structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-param destructive tool with no output schema, the description covers the mutations, the reversion guarantee, what is returned (diff, confidence), and the refusal case. It leaves auth/session params (headers, storageState) and config/timeout handling to the schema, which is acceptable given their 90% coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 90%, so the baseline is 3, but the description meaningfully extends key params: it explains that apply:true triggers a write plus verification with rollback, and that url heals against the live page instead of the trace snapshot — detail beyond the schema's terse 'default: diff only' and 'Heal against this live page'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource with scope: 'Patch a failure from the last run', and details the two concrete repair modes (drifted locator remap, changed expected text). This clearly separates it from siblings like get-failure (read) and run-test (execute).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context for use ('failure from the last run') and an explicit exclusion boundary ('Refuses when the element is truly missing (app bug)'), plus a variant path (live url vs trace). It does not name alternative siblings such as validate-selector or inspect-page for pre-healing analysis, so it stops short of full alternative routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate-selectorValidate selector or locatorARead-onlyIdempotentInspect
Check a CSS selector, Playwright selector or locator (getByRole/getByTestId…, .first(), .filter()) on a live page: validity, match count (actions need exactly 1), sample matches and a sturdier verified locator for each. Accepts storageState/headers/actions.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Full URL of the live page to test against (http/https) | |
| actions | No | Steps run after load to reach a state (modal, step 3) | |
| headers | No | Extra HTTP headers, e.g. Authorization | |
| selector | Yes | Selector or locator, e.g. getByRole('button', { name: 'Save' }) | |
| viewport | No | ||
| timeoutMs | No | Overall limit in ms (default 45000) | |
| projectRoot | No | Project whose Playwright launches the browser (default: server root) | |
| storageState | No | Logged-in session: storageState JSON inside the project, e.g. playwright/.auth/user.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/idempotentHint/openWorldHint, so the description earns credit for adding real context: it needs a live, possibly authenticated page (storageState/headers) and can be driven through intermediate states via actions. It stops short of noting the mild tension that 'actions' can click/fill the live page while the tool is annotated read-only, or any rate/timeout behavior beyond what the schema states.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the verb and fully packed into two sentences with no filler; every clause (locator kinds, live page, output list, accepted inputs) earns its place. The parenthetical locator examples make it dense but readable rather than padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter tool with nested action objects and no output schema, the description usefully enumerates what the call returns, which compensates for the absent output schema. The remaining gaps (defaults, pagination-style limits, behavior when zero or many matches occur) are minor and partly covered by the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is already 88%, so the baseline is 3, and the description goes slightly beyond by calling out storageState, headers and actions as accepted inputs and explaining the selector semantics with concrete locator syntax. It does not clarify defaults for timeoutMs or projectRoot, but the schema covers those.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (validate/check) and resource (CSS selector, Playwright selector or locator) plus the exact scope: against a live page. It enumerates the outputs the agent gets back (validity, match count, sample matches, sturdier verified locator), which cleanly distinguishes it from siblings like inspect-page or run-test that operate on whole pages or suites.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical 'actions need exactly 1' gives an implied usage rule (validate before acting), and 'on a live page' signals the required context, but there is no explicit when-to-use/when-not statement and no sibling is named as the alternative. The agent must infer that this is the pre-flight check rather than, say, run-test or suggest-fix.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
11 tool updates
v0.2.2- First observed
analyze-history - First observed
compare-visual-state - First observed
diagnose-flaky - First observed
generate-e2e-test - First observed
get-failure - First observed
get-run-status - First observed
inspect-page - First observed
list-tests - First observed
run-test - First observed
suggest-fix - First observed
validate-selector
TDQS
Scored across 11 tools
Tools map to distinct stages of the Playwright workflow: listing, running, monitoring, failure analysis, fixing, generation, visual comparison, selector validation, flaky diagnosis, and history. Overlap exists between diagnose-flaky and analyze-history, and between get-failure and get-run-status, but descriptions clarify the boundaries.
All tool names use kebab-case with a consistent verb-first pattern (run-test, get-failure, list-tests, validate-selector, etc.). Minor variation in noun length does not break the convention.
11 tools is well within the ideal 3–15 range. Each tool addresses a distinct aspect of Playwright E2E testing without obvious redundancy.
The core lifecycle is well covered: list tests, run tests, monitor runs, analyze failures, suggest fixes, generate tests, validate selectors, compare visual state, diagnose flakiness, and analyze history. Minor gaps include running a specific test or reading a single test's code, but agents can work around these.
Maintenance
Related MCP Connectors
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
Agentic AI quality engineering: run, debug and triage tests on real browsers and devices
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Discover Playwright workflows, start runs, and inspect results in Playrunner Cloud.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnables AI assistants to inspect, debug, and test web pages using Playwright. Provides comprehensive DOM inspection, visibility debugging, layout validation, and element finding capabilities in real browser environments.3470 npm5MIT
- AlicenseNot gradedqualityCmaintenanceEnables automated end-to-end testing powered by Playwright where test cases are defined in natural language and executed by AI. Uses lightweight snapshot analysis with vision mode fallback for sophisticated testing scenarios.3Apache 2.0
- AlicenseAqualityDmaintenanceRun real Playwright E2E tests from your AI coding agent.43 npmMIT
- FlicenseAqualityDmaintenanceEnables AI agents to run reusable Playwright test fixtures against live deployments, providing structured test results, screenshots, and assertions.3-