Ein Model Context Protocol-Server, der einer KI-Assistenz einen kleinen Satz konkreter QA-/Testanalyse-Werkzeuge an die Hand gibt, statt sie raten zu lassen: einen Playwright-Lauf zusammenfassen, flaky Tests über mehrere Läufe finden, einen Fehlschlag heuristisch erklären, ein Playwright-Testgerüst erzeugen und den aktuellen CI-Status eines GitHub-Repos abrufen. Richtet man Claude Desktop, Claude Code oder einen anderen MCP-kompatiblen Host auf den Server aus, werden daraus echte Tool-Aufrufe statt Freitext-Vermutungen.
Es ist das „Gib der KI echte Daten, keine Bauchgefühle"-Stück eines QA-Automations-Portfolios: Die anderen Repos dieses Portfolios (healthcare-qa-automation-framework, healthcare-data-quality-framework) erzeugen Playwright-JSON-Reports und CI-Läufe; dieser Server macht daraus etwas, das eine Assistentin bzw. ein Assistent tatsächlich abfragen kann.
Die Protokoll-Anbindung ist eine dünne Schicht über gewöhnliche Funktionen. Die eigentliche Logik jedes Tools lebt in src/playwright/, src/github/ und src/tools/ als kleine, weitgehend pure, unabhängig unitgetestete Module. src/index.ts registriert sie nur mit dem MCP-SDK – der fehleranfällige Teil (Report-Parsing, Flaky-Erkennung, Heuristiken) ist also durch schnelle Unit-Tests abgedeckt, die Protokoll-Verdrahtung durch einen separaten End-to-End-Smoke-Test (weiter unten).
GitHub-Zugriff ist pluggbar – nach demselben Muster wie die LLM-Integrationen im übrigen Teil dieses Portfolios. GitHubClient ist ein Interface; RealGitHubClient kapselt fetch gegen die GitHub-REST-API, FakeGitHubClient liefert vorgefertigte Daten. Dadurch ist get_repo_ci_status offline vollständig unit-testbar, während der echte Client echter, lauffähiger Code ist – ein MCP-Host auf dem Rechner einer Entwicklerin oder eines Entwicklers hat ganz normalen Internetzugang, selbst in Umgebungen, die keinen haben (wie der eigenen CI-Sandbox dieses Repos).
Die Teststub-Generierung ist bewusst deterministisch und nicht LLM-basiert. Bei einer benannten Sequenz von data-testid-Interaktionen erzeugt generate_playwright_test_stub immer genau dieselbe Ausgabe – sofort, kostenlos, reproduzierbar. Für einen anderen Zweck gibt es ein eigenes Projekt: Testfälle aus einer Spezifikation oder User Story zu erzeugen, wo ein LLM tatsächlich Mehrwert bringt.
Ein Smoke-Test auf Protokollebene, nicht nur Unit-Tests. smoke-test.mjs startet den gebauten Server als echten Kindprozess und steuert ihn mit dem offiziellen MCP-Client über stdio – tools/list, dann ein tools/call pro Tool. Damit beweist CI das einen echten Wire-Protokolle ebenso wie nur die Funktionen dahinter.
Related MCP server: Playwright Debug MCP Server
Tool | Was es tut |
| Parsiert Playwright-JSON-Reporter-Output in Pass-/Fail-/Timeout-/Skipped-/Flaky-Zähler, Dauer und vollständige Fehlerdetails. |
| Vergleicht dieselben Tests über 2+ Playwright-JSON-Reports und markiert alle, deren Ausgang nicht einheitlich war. |
| Heuristische, musterbasierte Vermutung zur Ursache einer rohen Fehlermeldung. Vollständig offline. |
| Baut deterministisch ein lauffähiges Playwright-Testgerüst aus einer benannten Sequenz von |
| Holt die aktuellen Actions-Läufe eines öffentlichen GitHub-Repos und zieht eine Bilanz aus Pass-/Fail-/In-Progress-Zahlen. |
src/
index.ts MCP server: registers all 5 tools, connects over stdio
playwright/reportParser.ts Pure Playwright-JSON parsing: summarizeReport, findFlakyTests
github/githubClient.ts GitHubClient interface + RealGitHubClient (fetch) + FakeGitHubClient
tools/
explainFailure.ts Heuristic pattern-matching (ported from the triage_failure.py
script in healthcare-qa-automation-framework)
generateTestStub.ts Deterministic Playwright test-file template generator
getRepoCiStatus.ts Wraps GitHubClient into pass/fail/in-progress counts
test/ Vitest unit tests for every module above
fixtures/ Sample Playwright JSON reports (3 runs, used for flaky-test tests)
smoke-test.mjs End-to-end MCP protocol smoke test (real child process, real client)
.github/workflows/ci.yml Type-check, unit tests, build, protocol smoke testTypeScript · Node.js · @modelcontextprotocol/sdk · Zod · Vitest · GitHub REST API ·
GitHub Actions
npm install
npm run build # compiles to dist/
npm test # unit tests (27 tests, fully offline)
npm run smoke # builds, then drives the real server over stdio via the MCP clientRichten Sie einen beliebigen MCP-kompatiblen Client auf den gebauten Server, z. B. in Claudes
claude_desktop_config.json:
{
"mcpServers": {
"qa-copilot": {
"command": "node",
"args": ["/absolute/path/to/qa-copilot-mcp-server/dist/index.js"]
}
}
}Setzen Sie GITHUB_TOKEN in der Umgebung, um das unauthentifizierte GitHub-Ratenlimit für get_repo_ci_status anzuheben (60 Anfragen pro Stunde ohne Token, 5.000 pro Stunde mit Token) – für gelegentliche Nutzung funktioniert es auch ganz ohne Token.
Jeder Push und Pull-Request auf main läuft einen einzigen Job: Typprüfung (tsc --noEmit), die Unit-Test-Suite (vitest run), ein Produktions-Build und den Smoke-Test auf Protokoll-Ebene (node smoke-test.mjs) gegen die gebaute Ausgabe. CI setzt nirgendwo Netzwerkzugang voraus oder benötigt ihn – get_repo_ci_status wird in den Unit-Tests nur gegen FakeGitHubClient ausgeführt, und der Smoke-Test wertet ein netzwerkfreies Ergebnis von get_repo_ci_status als Erfolg, solange der Server es sauber über den isError-Kanal des MCP weitergibt statt abzustürzen.
Das ist kein Allzweck-GitHub- oder CI-Dashboard – es ist ein kleine, fokussierte Sammlung von Tools, die gezielt für die Playwright-Berichts- und GitHub-Actions-Formate gebaut wurde, die die anderen Repos dieses Portfolios tatsächlich erzeugen. Es soll zeigen, wie eine KI-Assistenz mit realen, überprüfbaren Testdaten verbunden wird, statt von ihr zu verlangen, Testergebnisse aus einer Paraphrase zu erraten.
Given a Playwright (or similar) error message, returns a heuristic, pattern-matched guess at the root cause and a suggested next step. Fully offline - no LLM call.
| Name | Required | Description | Default |
|---|---|---|---|
| errorMessage | Yes | The raw error/assertion message from a failed test. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since there are no annotations, the description carries the full burden of behavioral disclosure. It does well by stating that the output is a 'guess', that the analysis is 'heuristic, pattern-matched', and that it is 'Fully offline - no LLM call'. This conveys uncertainty and operational constraints clearly, though a bit more detail about output format or confidence would have made it even richer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences accomplish all necessary tasks: defining input, action, output, and an important operational constraint. The key behavioral details are front-loaded, with no wasted or repetitive wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter, absent annotations, and no output schema, the description is nearly complete: it defines the input, the output type (root cause and next step), and the heuristic/offline behavior. The only minor gap is a precise description of the return value's shape, but for this simple tool the current wording is sufficient for an agent to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema's description of errorMessage already covers 'raw error/assertion message from a failed test'. The tool description adds value by scoping the input to a 'Playwright (or similar) error message', which clarifies the expected domain. With exceptionally high schema coverage at the file level, the description's incremental input is small but relevant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies a specific verb ('returns') and a distinct resource ('a heuristic, pattern-matched guess at the root cause and a suggested next step'). The input is scoped to 'Playwright (or similar) error messages', making it easy to differentiate from sibling tools like summarize_test_run or list_flaky_tests. An agent can understand this tool's purpose without inferring anything.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The opening 'Given a Playwright (or similar) error message' clearly indicates when this tool should be used. However, the description does not explicitly name sibling tools as exclusions or state when not to use it, so the comparison with alternatives is left to the agent's own judgment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Deterministically scaffolds a runnable Playwright test file from a named sequence of data-testid interactions (fill, click, expectText, expectVisible). Template-based, not LLM-based - instant and 100% reproducible.
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| testName | Yes | The test's title, e.g. "logs in with valid credentials". | |
| describeName | No | The test.describe() block title. Defaults to "Generated suite". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description does a solid job of setting expectations: it is template-based, deterministic, instant, reproducible, and produces a runnable file. It does not disclose whether the file is written to disk or returned as text, but the key behavioral traits for agent decision-making are clearly surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, concise and front-loaded. It states the action/resource first, then adds the relevant constraints (deterministic, template-based, reproducible). No sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a code generation tool with three parameters and no output schema, the description communicates enough for most invocation decisions: inputs are structured steps, the output is a runnable Playwright test stub, and generation is deterministic. The main missing piece is whether 'scaffolds' means writing a file, returning code, or both, which could affect how the agent uses or verifies the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents testName and describeName, covering about 67% of parameters. The description adds minor value by characterizing steps as a named sequence of data-testid interactions and enumerating the action types, but it does not provide additional meaning about testName or describeName beyond the schema. The nested schema definition of steps already covers most of the needed detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('scaffolds') with a clear resource ('a runnable Playwright test file') and defines the input as a sequence of data-testid interactions with named action types. It also differentiates itself from analysis-focused siblings like summarize_test_run and explain_failure by emphasizing instant, deterministic generation rather than interpretation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used to generate a Playwright test stub when you have a deterministic sequence of data-testid interactions, and emphasizes that it is not LLM-based, which rules out ad-hoc/creative generation use cases. It does not explicitly name alternatives or exclusion conditions, but no sibling tool competes for the same role.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetches the last N GitHub Actions workflow runs for a public repo (owner/repo) and summarizes pass/fail/in-progress counts alongside the raw run list. Requires network access from wherever this server is running; set GITHUB_TOKEN in the environment to raise the unauthenticated rate limit.
| Name | Required | Description | Default |
|---|---|---|---|
| repo | Yes | Repo name, e.g. "healthcare-qa-automation-framework". | |
| limit | No | How many recent runs to fetch (default 10). | |
| owner | Yes | Repo owner, e.g. "sharath9271-design". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description bears the full burden, and it adds real value: it reveals that the tool needs network access, only works on public repos, and can raise its rate limit by setting GITHUB_TOKEN. 'Fetches' implies a read-only operation, and the description does not conceal any dangerous or surprising side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core fetching and summarizing behavior, followed by concise operational caveats. Nothing feels like filler; both sentences earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, three-parameter tool with no output schema, the description covers everything essential: what is fetched, what the response contains, public-repo scope, network requirements, and the optional auth token. An agent has enough context to decide whether to reply and how to handle it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all three parameters (owner, repo, limit) with types, explanations, and examples, covering 100% of its parameters. The description merely restates the limit in prose as 'last N', so it does not add meaningful parameter semantics beyond what the schema conveys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb-resource pair: 'Fetches the last N GitHub Actions workflow runs' and 'summarizes pass/fail/in-progress counts alongside the raw run list'. This clearly distinguishes it from the sibling test-analysis tools, which talk about tests, flakes, and failures rather than repo-level CI status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied and fairly obvious: an agent would call it when asked for recent CI status of a public repo. However, it never names a sibling alternative or states when this tool is not appropriate, so routing is left to inference rather than explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compares the same tests across two or more Playwright JSON reports (e.g. the last N CI runs of the same suite) and returns every test whose outcome wasn't consistent - it passed in some runs and failed/timed out in others.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJsons | Yes | Two or more Playwright JSON reporter outputs, oldest first, as raw strings. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description correctly conveys that this is a read-only comparison and what task qualifies as flaky. However, it does not disclose how missing tests across reports are treated, whether the result is influenced by report ordering beyond the schema note, or what fields are returned per test — leaving some behavioral details implied rather than explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence that states the operation, the input shape, the example use, and the result. There is no filler, and the purpose appears immediately. It is concise despite the long sentence because every clause adds new information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter query tool with no output schema, the description explains the input to the comparison, what counts as 'flaky', and what the output represents. It omits edge-case details, such as how test records without entries in all reports are handled or the exact shape of each returned test, but the overall information is sufficient for an agent to invoke it correctly for a typical flaky-test scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents reportJsons as raw Playwright JSON reporter outputs, oldest first, with minItems=2 (100% schema coverage). The description adds the meaningful context that these should be 'the last N CI runs of the same suite', which clarifies the intended input beyond the bare schema definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb-resource pair ('Compares the same tests across two or more Playwright JSON reports') and explicitly names the result ('returns every test whose outcome wasn't consistent'). It clearly differentiates this from siblings like get_repo_ci_status or explain_failure by focusing on flaky test detection across multiple runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides contextual use ('two or more... e.g. the last N CI runs of the same suite') and makes the minimum-input condition explicit. It does not spell out 'when not to use' or name alternatives, but the multi-report scope plus sibling tool context makes the boundary reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Parses Playwright's JSON reporter output and returns pass/fail/timeout/skipped/flaky counts, total duration, and full details on every failure.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJson | Yes | The raw contents of a Playwright JSON reporter output file (results.json). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full burden; it handles this well by describing what the tool does: parse JSON and return counts, duration, and failure details. It doesn't discuss edge cases, output formatting, or potential errors, but for a straightforward parsing and summarization tool this is a substantially clear behavioral explanation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately states the verb, the input, and the exact outputs. It is front-loaded with 'Parses...' and contains no filler or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, conceptually simple tool, the description fully covers what the input is (Playwright JSON reporter output), what the tool produces, and how the result will be structured. No output schema is present, but the return values are described in enough detail for an agent to consume the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter is already fully documented—'The raw contents of a Playwright JSON reporter output file (results.json)'. The tool description adds no additional parameter-level meaning beyond re-stating that it parses Playwright JSON, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear action ('parses Playwright's JSON reporter output') and reports a specific set of results: pass/fail/timeout/skipped/flaky counts, total duration, and failure details. This distinguishes it from sibling tools like list_flaky_tests or explain_failure, which have narrower or different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended context is clear: this tool is used when a Playwright JSON reporter output is available and a summary of counts, duration, and failures is needed. It does not explicitly mention when not to use it or compare itself with sibling tools, but the usage context is strongly implied and unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0explain_failuregenerate_playwright_test_stubget_repo_ci_statuslist_flaky_testssummarize_test_runScored across 5 tools
Each tool targets a distinct responsibility—summarizing test output, identifying flaky tests, explaining failures, generating test stubs, and fetching CI status. There is no meaningful overlap between any pair of tools.
All tool names follow a consistent verb_noun pattern with clear action-prefixed names: summarize_, list_, explain_, generate_, get_. Minor pluralization differences do not create confusion.
Five tools cover a tightly scoped QA workflow—analyze results, detect flakiness, explain failures, generate tests, and check CI—without redundancy or bloat.
The core QA reporting and triage loop is well supported, from parsing reports to explaining failures and detecting flaky tests. Minor gaps like actionable test-edit or test-deletion operations are absent, but they fall outside the visible purpose.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
Create and manage test cases, test runs and results in Testiny from your AI assistant.
