Skip to main content
Glama
baramubarok

playwright-generate-mcp

by baramubarok

Run Playwright test

run_playwright_test

Runs a Playwright spec file and returns structured test results with status, failure details, diagnostics, and artifact paths for E2E debugging.

Instructions

Run a Playwright spec file in the target project and return a bounded, structured result: explicit status, counts, runner/process metadata, up to 5 failures with location, error type, recent steps, browser console errors, page errors and failed requests (from trace.zip and/or the scaffolded diagnostics fixture), screenshot/trace/video/error-context paths, and a compact qualityScore. The project's own reporters (HTML, Allure) keep running. Use detail:'full' or get_failure_details for more. TOKEN ECONOMY INSTRUCTION: In your response, provide a clean, compact 2–3 line summary (status, duration, video link, and failure cause) rather than dumping large markdown tables into chat. If tests fail, analyze the error and formulate a concrete fix proposal in '.page.ts'. To confirm with the user before applying fixes: 1) If an interactive question/selection tool is available in your environment (e.g. ask_question, vscode_askQuestions), MUST use it to present options: ['Apply Fix & Re-run', 'Review Diff First', 'Cancel']. 2) Otherwise, present the options as a clean numbered list in your response so the user can easily reply with just the number.

Input Schema

NameRequiredDescriptionDefault
grepNoOnly run tests whose title matches this pattern (passed to --grep).
detailNoResponse size. 'summary' (default) returns status, counts, up to 5 failures with bounded diagnostics and artifact paths. 'full' adds every test with retry attempts, runner stdout/stderr and the complete quality score. Prefer get_failure_details for one test.
headedNoRun in headed (visible browser) mode instead of headless.
specPathYesPath to the spec file, relative to project root or absolute, e.g. "e2e/payment.spec.ts".
timeoutMsNoMaximum runner duration in milliseconds (default 120000; allowed range 100–900000).
projectRootNoOptional path to the target project root. If omitted, auto-detected by searching for playwright.config.*.
generateHtmlReportNoSet to true to generate the standalone visual quality report at test-results/quality-report.html.

Output Schema

NameRequiredDescriptionDefault
testsYes
detailYes
runnerNo
statusYes
summaryYes
failuresNo
videoPathNo
durationMsYes
reportPathNo
projectRootYes
runnerErrorNo
outputBudgetNo
qualityScoreNo
testsOmittedNo
traceZipPathNo
nativeReportsNo
prerequisitesNo
failuresOmittedNo
healingGateDirectiveNo
qualityReportHtmlPathNo
reportPromptDirectiveNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does disclose real behavior: results are bounded, up to 5 failures are returned, artifact paths and diagnostics come from trace.zip/diagnostics fixture, and the project's own reporters keep running. It does not disclose side effects of executing a test run (process spawn, timeouts, server/runner requirements) or what happens on timeout, and a large share of the text describes the agent's chat response rather than the tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and clear, but roughly half the text is an off-topic 'TOKEN ECONOMY INSTRUCTION' plus a three-option fix-confirmation protocol that describes how the agent should format its reply rather than what the tool does. Those sentences do not earn their place in a tool definition and dilute the signal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be re-explained, and the description still covers run scope, filtering, timeout, reporting and artifact generation adequately for a 7-parameter tool. The remaining gap is environment/prerequisite context, though sibling tools (check_test_prerequisites, check_playwright_config, set_project_root) partially cover that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with per-parameter descriptions for all 7 parameters, including the detail enum semantics, so the schema does the heavy lifting. The description only echoes the detail:'full' behavior, adding no new parameter meaning. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Run a Playwright spec file in the target project') and immediately characterizes the return shape. It differentiates itself from the sibling get_failure_details by naming it as the route for deeper per-test detail. An agent can identify the tool's role without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Routes the agent to two alternatives ('Use detail:'full' or get_failure_details for more'), which is genuine when-to-use guidance. However, it never states prerequisites (e.g., that check_test_prerequisites / project root setup may be needed) or when *not* to run this tool, and the guidance is embedded in a large block of unrelated workflow instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.