A
licenseB
qualityD
maintenanceSupercharges AI-assisted debugging of Playwright tests by parsing trace files to extract failures, action history, network logs, screenshots, and suggesting fixes.
6
3 npm
MIT
Playwright Report MCP
by nora-weisser
Playwright Report MCP
Playwright JSON 테스트 리포트를 LLM 에이전트용 도구로 노출하는 MCP 서버입니다.
요구 사항
Python >= 3.10
uv (
brew install uv또는curl -LsSf https://astral.sh/uv/install.sh | sh)Node.js + npx — MCP Inspector를 실행하는
mcp dev에만 필요합니다
Related MCP server: QA Copilot MCP Server
1. 의존성 설치
저장소 루트에서:
uv sync이렇게 하면 .venv/가 생성되고 프로젝트가 편집 가능(editable) 모드로 설치됩니다. 아래의 모든 명령은 uv run을 통해 실행되므로 venv를 수동으로 활성화할 필요가 없습니다.
2. Playwright JSON 리포트 생성
서버는 json 리포터가 생성한 Playwright 리포트를 읽습니다. Playwright 프로젝트에서:
npx playwright test --reporter=json > results.json또는 playwright.config.ts에 영구적으로 구성합니다:
export default defineConfig({
reporter: [['json', { outputFile: 'results.json' }]],
});파일에는 표준 최상위 키인 config, suites, errors, stats가 포함되어야 합니다.
서버를 간단히 사용해 보려면 샘플 리포트가 results.json에 포함되어 있습니다.
3. 리포트 경로 구성
저장소 루트에 해당 리포트를 가리키는 .env 파일을 생성합니다(절대 경로 권장):
REPORT_PATH=/absolute/path/to/results.json대신 포함된 샘플을 사용하려면:
REPORT_PATH=/absolute/path/to/playwright-test-analysis-mcp/src/playwright_report_mcp/results.jsonREPORT_PATH는 모든 도구 호출 시 읽히므로 서버를 재시작하지 않고도 리포트를 다시 생성할 수 있습니다. 설정되지 않은 경우 도구 호출은 REPORT_PATH environment variable is not set. 오류로 실패합니다.
4. 서버 실행
대화형 개발(MCP Inspector)
uv run mcp dev src/playwright_report_mcp/server.py브라우저에서 MCP Inspector를 열어 도구를 직접 나열하고 호출할 수 있습니다. npx가 필요합니다.
일반 stdio 서버
uv run playwright-report-mcp또는 동일하게:
uv run mcp run src/playwright_report_mcp/server.pystdin/stdout을 통해 MCP 프로토콜을 사용합니다. MCP 클라이언트에 연동하거나 빠른 스모크 테스트에 유용합니다:
printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"t","version":"1"}}}\n' \
| uv run playwright-report-mcp"serverInfo":{"name":"Playwright Report",...}가 포함된 JSON-RPC 결과를 받게 됩니다.
클라이언트에 연결
Claude Desktop에 서버를 등록합니다:
uv run mcp install src/playwright_report_mcp/server.py또는 MCP 클라이언트 구성(예: claude_desktop_config.json)에 수동으로 추가합니다:
{
"mcpServers": {
"playwright-report": {
"command": "uv",
"args": [
"--directory",
"/absolute/path/to/playwright-test-analysis-mcp",
"run",
"playwright-report-mcp"
]
}
}
}--directory는 중요합니다: uv가 이 프로젝트의 venv를 인식하게 하고 load_dotenv()가 .env 파일을 찾게 해줍니다.
도구
도구 | 설명 |
| 최신 실행 요약: |
| 실패한 테스트 목록. 각 테스트에는 |
테스트 실행
pytest와 pytest-asyncio가 설치되어 있지만 저장소에 아직 테스트 파일이 없으므로 현재 0개의 테스트가 수집됩니다:
uv run pytest문제 해결
증상 | 해결 방법 |
| 저장소 루트에 |
도구 호출 시 |
|
| JSON이 Playwright |
|
|
Available Tools
2 toolsget_failuresA
Return failed tests of the latest Playwright test run.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
A3.8/5.0
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It conveys read-only behavior via 'Return' and adds the useful 'latest run' scope, but it does not disclose behavior such as how empty results are handled or whether failures are ordered or grouped. These gaps are minor for a simple parameterless read and are partly covered by the output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that front-loads the action and object and contains no filler. Every word contributes to selection and invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the zero-parameter complexity and the presence of an output schema, the description is nearly complete for correct tool selection and invocation. The only material gap is the lack of any interaction with the sibling get_test_summary, but that is a usage-guidance concern rather than a completeness failure for this simple read.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is no parameter semantics burden for the description to carry. The baseline of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear action ('Return'), a precise resource ('failed tests'), and a scoping qualifier ('latest Playwright test run'). It effectively distinguishes itself from get_test_summary by focusing on failures rather than a summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit guidance about when to use this tool versus get_test_summary. It implies only that results are scoped to the latest run, but does not state when to prefer failures over a summary or whether get_test_summary is the alternative for successful tests.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_test_summaryA
Return summary of the latest Playwright test run.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| flaky | Yes | |
| failed | Yes | |
| passed | Yes | |
| skipped | Yes | |
| start_time | Yes | |
| duration_ms | Yes |
TDQS
A3.5/5.0
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the behavioral disclosure burden. The word 'Return' suggests a read-only operation, but the description does not mention what happens when no test run exists, whether the summary is limited by filters, or any side effects. This is minimally adequate but lacks richer behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, compact sentence that front-loads the action ('Return') and the object ('summary') with no filler or unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter retrieval tool, the description is sufficiently complete: the resource is identified, the action is clear, and the presence of an output schema covers return-value details. The main gap is lack of usage guidance relative to the sibling, but that is a separate dimension.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the input schema is trivially complete. The description does not need to explain parameter semantics, and the baseline of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Return') and the resource ('summary of the latest Playwright test run'). It distinguishes from the sibling 'get_failures' by focusing on the overall summary rather than failures, but it does not explicitly name or contrast the sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus 'get_failures' or any other alternative. The description implies it is for retrieving a summary, but it does not state exclusion criteria, prerequisites, or when another tool would be preferable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
2 tool updates
v0.1.0- First observed
get_failures - First observed
get_test_summary
TDQS
A3.8/5.0
Scored across 2 tools
Disambiguation5/5
get_failures and get_test_summary have clearly distinct purposes: one retrieves only failed tests, the other an aggregate summary. There is no overlap or ambiguity between them.
Naming Consistency5/5
Both tools follow a consistent get_[noun] pattern, making the naming predictable and easy to infer. The convention is uniform across the entire set.
Tool Count3/5
With only two tools, the set is on the thin side and falls into the borderline range. The tools are focused and justified, but the count is minimal for a server that could reasonably include more report-oriented operations.
Completeness4/5
For a server scoped to the latest Playwright test run, get_failures and get_test_summary cover the primary reporting needs. Minor gaps exist—such as retrieving test details or run history—but they do not create dead ends for the core use case.
Maintenance
ActivityMaintained
ResponsivenessSyncing
Related MCP Connectors
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
Discover Playwright workflows, start runs, and inspect results in Playrunner Cloud.
Ingest and search LogsLoom logs from coding agents.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnables AI assistants to summarize Playwright test runs, identify flaky tests, heuristically triage failures, generate Playwright test stubs, and retrieve recent GitHub CI status through MCP tool calls.5MIT
failure-reportofficial
AlicenseNot gradedqualityBmaintenanceEnables querying Playwright test failure reports and execution results directly from AI assistants, allowing retrieval of test outcomes, failures, and report details.MIT- AlicenseNot gradedqualityAmaintenanceEnables coding agents to validate web pages and run repeatable JSON browser scenarios, returning pass/fail results with local Playwright evidence such as screenshots, traces, and JUnit XML.6 npm1Apache 2.0