Skip to main content
Glama
Sign In
deenesjakoruzh
Playwright Report MCP

Playwright Report MCP

by nora-weisser
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Playwright Report MCP

Ein MCP-Server, der Playwright-JSON-Testberichte als Tools für LLM-Agenten bereitstellt.

Voraussetzungen

  • Python >= 3.10

  • uv (brew install uv oder curl -LsSf https://astral.sh/uv/install.sh | sh)

  • Node.js + npx — nur für mcp dev, das den MCP Inspector startet

Related MCP server: QA Copilot MCP Server

1. Abhängigkeiten installieren

Vom Repository-Wurzelverzeichnis aus:

uv sync

Dadurch wird .venv/ erstellt und das Projekt im bearbeitbaren Modus installiert. Jeder unten aufgeführte Befehl wird über uv run ausgeführt, sodass Sie die venv nie manuell aktivieren müssen.

2. Einen Playwright-JSON-Bericht erzeugen

Der Server liest einen Playwright-Bericht, der vom json-Reporter erzeugt wurde. In Ihrem Playwright-Projekt:

npx playwright test --reporter=json > results.json

Oder konfigurieren Sie es dauerhaft in playwright.config.ts:

export default defineConfig({
  reporter: [['json', { outputFile: 'results.json' }]],
});

Die Datei muss die standardmäßigen Top-Level-Schlüssel config, suites, errors, stats enthalten.

Ein Beispielbericht ist unter results.json enthalten, falls Sie den Server nur ausprobieren möchten.

3. Berichtspfad konfigurieren

Erstellen Sie im Repository-Wurzelverzeichnis eine .env-Datei, die auf diesen Bericht verweist (absoluter Pfad empfohlen):

REPORT_PATH=/absolute/path/to/results.json

Um stattdessen das mitgelieferte Beispiel zu verwenden:

REPORT_PATH=/absolute/path/to/playwright-test-analysis-mcp/src/playwright_report_mcp/results.json

REPORT_PATH wird bei jedem Tool-Aufruf gelesen, sodass Sie den Bericht neu erzeugen können, ohne den Server neu zu starten. Wenn sie nicht gesetzt ist, schlagen Tool-Aufrufe mit REPORT_PATH environment variable is not set. fehl.

4. Server ausführen

Interaktive Entwicklung (MCP Inspector)

uv run mcp dev src/playwright_report_mcp/server.py

Öffnet den MCP Inspector in Ihrem Browser, wo Sie die Tools manuell auflisten und aufrufen können. Erfordert npx.

Einfacher stdio-Server

uv run playwright-report-mcp

oder, äquivalent:

uv run mcp run src/playwright_report_mcp/server.py

Spricht das MCP-Protokoll über stdin/stdout. Nützlich zum Anbinden an einen MCP-Client oder für einen kurzen Smoke-Test:

printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"t","version":"1"}}}\n' \
  | uv run playwright-report-mcp

Sie sollten ein JSON-RPC-Ergebnis mit "serverInfo":{"name":"Playwright Report",...} zurückerhalten.

Mit einem Client verbinden

Registrieren Sie den Server bei Claude Desktop:

uv run mcp install src/playwright_report_mcp/server.py

Oder fügen Sie ihn manuell zu einer MCP-Client-Konfiguration hinzu (z. B. claude_desktop_config.json):

{
  "mcpServers": {
    "playwright-report": {
      "command": "uv",
      "args": [
        "--directory",
        "/absolute/path/to/playwright-test-analysis-mcp",
        "run",
        "playwright-report-mcp"
      ]
    }
  }
}

--directory ist wichtig: Es bewirkt, dass uv die venv dieses Projekts auflöst, und lässt load_dotenv() die .env-Datei finden.

Tools

Tool

Beschreibung

get_test_summary

Zusammenfassung des letzten Laufs: start_time, duration_ms, passed, failed, skipped, flaky.

get_failures

Liste der fehlgeschlagenen Tests, jeweils mit title, file, status und der ersten error-Meldung.

Tests ausführen

pytest und pytest-asyncio sind installiert, aber das Repository hat noch keine Testdateien, daher werden derzeit 0 Tests gesammelt:

uv run pytest

Fehlerbehebung

Symptom

Behebung

REPORT_PATH environment variable is not set.

Erstellen Sie .env im Repository-Wurzelverzeichnis oder exportieren Sie REPORT_PATH in der Shell, die den Server startet.

FileNotFoundError bei einem Tool-Aufruf

REPORT_PATH verweist auf eine Datei, die nicht existiert — verwenden Sie einen absoluten Pfad.

KeyError: 'stats'

Die JSON-Datei ist kein Bericht des Playwright-json-Reporters (z. B. ein HTML-Bericht oder ein roher Blob).

mcp dev startet nicht

Installieren Sie Node.js, damit npx verfügbar ist, oder verwenden Sie stattdessen mcp run.

Available Tools

2 tools
get_failuresA

Return failed tests of the latest Playwright test run.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It conveys read-only behavior via 'Return' and adds the useful 'latest run' scope, but it does not disclose behavior such as how empty results are handled or whether failures are ordered or grouped. These gaps are minor for a simple parameterless read and are partly covered by the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that front-loads the action and object and contains no filler. Every word contributes to selection and invocation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the zero-parameter complexity and the presence of an output schema, the description is nearly complete for correct tool selection and invocation. The only material gap is the lack of any interaction with the sibling get_test_summary, but that is a usage-guidance concern rather than a completeness failure for this simple read.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no parameter semantics burden for the description to carry. The baseline of 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear action ('Return'), a precise resource ('failed tests'), and a scoping qualifier ('latest Playwright test run'). It effectively distinguishes itself from get_test_summary by focusing on failures rather than a summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit guidance about when to use this tool versus get_test_summary. It implies only that results are scoped to the latest run, but does not state when to prefer failures over a summary or whether get_test_summary is the alternative for successful tests.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_summaryA

Return summary of the latest Playwright test run.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
flakyYes
failedYes
passedYes
skippedYes
start_timeYes
duration_msYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral disclosure burden. The word 'Return' suggests a read-only operation, but the description does not mention what happens when no test run exists, whether the summary is limited by filters, or any side effects. This is minimally adequate but lacks richer behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, compact sentence that front-loads the action ('Return') and the object ('summary') with no filler or unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter retrieval tool, the description is sufficiently complete: the resource is identified, the action is clear, and the presence of an output schema covers return-value details. The main gap is lack of usage guidance relative to the sibling, but that is a separate dimension.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the input schema is trivially complete. The description does not need to explain parameter semantics, and the baseline of 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Return') and the resource ('summary of the latest Playwright test run'). It distinguishes from the sibling 'get_failures' by focusing on the overall summary rather than failures, but it does not explicitly name or contrast the sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus 'get_failures' or any other alternative. The description implies it is for retrieving a summary, but it does not state exclusion criteria, prerequisites, or when another tool would be preferable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv0.1.0
    • First observedget_failures
    • First observedget_test_summary
Deploy Server
Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

TDQS

A3.8/5.0

Scored across 2 tools

Disambiguation5/5

get_failures and get_test_summary have clearly distinct purposes: one retrieves only failed tests, the other an aggregate summary. There is no overlap or ambiguity between them.

Naming Consistency5/5

Both tools follow a consistent get_[noun] pattern, making the naming predictable and easy to infer. The convention is uniform across the entire set.

Tool Count3/5

With only two tools, the set is on the thin side and falls into the borderline range. The tools are focused and justified, but the count is minimal for a server that could reasonably include more report-oriented operations.

Completeness4/5

For a server scoped to the latest Playwright test run, get_failures and get_test_summary cover the primary reporting needs. Minor gaps exist—such as retrieving test details or run history—but they do not create dead ends for the core use case.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers