Un servidor Model Context Protocol que proporciona a un asistente de IA un pequeño conjunto de herramientas concretas de QA e inteligencia de pruebas, en lugar de pedirle que adivine: resumir una ejecución de Playwright, encontrar tests flaky entre varias ejecuciones, explicar un fallo de forma heurística, generar un stub de test de Playwright y consultar el estado reciente de CI de un repositorio de GitHub. Apunta a Claude Desktop, Claude Code o cualquier otro host compatible con MCP y esas capacidades pasan a ser llamadas reales a herramientas, no suposiciones en texto libre.
Es la pieza de «dale a la IA datos reales, no sensaciones» dentro de un portafolio de automatización de QA: los otros repositorios de este portafolio (healthcare-qa-automation-framework, healthcare-data-quality-framework) generan informes JSON de Playwright y ejecuciones de CI; este servidor es lo que convierte esos datos en algo que un asistente puede consultar de verdad.
El cableado del protocolo es una capa fina sobre funciones simples. La lógica real de cada herramienta vive en src/playwright/, src/github/ y src/tools/ como módulos pequeños, puros (o casi) y con pruebas unitarias independientes. src/index.ts solo los registra con el SDK de MCP, así que la parte complicada (el análisis de informes, la detección de flaky, las heurísticas) queda cubierta por pruebas unitarias rápidas, y la capa del protocolo la cubre una prueba de humo de extremo a extremo aparte (más abajo).
El acceso a GitHub is pluggable, con el mismo patrón que las integraciones de LLM del resto del portafolio. GitHubClient una interfaz; RealGitHubClient envuelve fetch contra la GitHub REST API, y FakeGitHubClient devuelve datos fijos. Eso hace que get_repo_ci_status sea totalmente testeable unitariamente sin red, mientras que el cliente real es código genuino y ejecutable: un host MCP en la máquina de un desarrollador tiene acceso normal a internet, incluso en entornos (como el sandbox de CI de este mismo repositorio) que no lo tienen.
La generación de stubs de pruebas es determinista y a propósito no se base en un LLM. Dada una secuencia con nombre de interacciones data-testid, generate_playwright_test_stub siempre produce exactamente la misma salida: instantánea, libremente reproducible. Una herramienta distinta para un trabajo distinto (generar casos de prueba a partir de una especificación o una historia de usuario, donde un LLM sí aporta valor) es un proyecto aparte dentro de este portafolio.
Una prueba de humo a nivel de protocolo, no solo unit tests. smoke-test.mjs lanza el servidor compilado como un proceso hijo real y lo conduce con el Client oficial de MCP a través de stdio —tools/list y luego un tools/call por cada herramienta—, de modo que CI comprueba que el protocolo de red real funciona, no solo las funciones que hay detrás.
Related MCP server: Playwright Debug MCP Server
Herramienta | Qué hace |
| Convierte la salida del reporter JSON de Playwright en conteos de pass/fail/timeout/skipped/flaky, duración y toda la información del fallo. |
| Compara las mismas pruebas en 2+ informes JSON de Playwright y señala cualquier prueba cuyo resultado no haya sido consistente. |
| Estimación heurística de causa raíz mediante plantillas que coinciden con el mensaje de error crudo. Totalmente offline. |
| Genera de forma determinista un test Playwright ejecutable a partir de una secuencia nombrada de interacciones |
| Obtiene las ejecuciones recientes de Actions de un repositorio público de GitHub y resume los conteos de éxito/fallo/en curso. |
src/
index.ts MCP server: registers all 5 tools, connects over stdio
playwright/reportParser.ts Pure Playwright-JSON parsing: summarizeReport, findFlakyTests
github/githubClient.ts GitHubClient interface + RealGitHubClient (fetch) + FakeGitHubClient
tools/
explainFailure.ts Heuristic pattern-matching (ported from the triage_failure.py
script in healthcare-qa-automation-framework)
generateTestStub.ts Deterministic Playwright test-file template generator
getRepoCiStatus.ts Wraps GitHubClient into pass/fail/in-progress counts
test/ Vitest unit tests for every module above
fixtures/ Sample Playwright JSON reports (3 runs, used for flaky-test tests)
smoke-test.mjs End-to-end MCP protocol smoke test (real child process, real client)
.github/workflows/ci.yml Type-check, unit tests, build, protocol smoke testTypeScript · Node.js · @modelcontextprotocol/sdk · Zod · Vitest · GitHub REST API ·
GitHub Actions
npm install
npm run build # compiles to dist/
npm test # unit tests (27 tests, fully offline)
npm run smoke # builds, then drives the real server over stdio via the MCP clientApunta cualquier cliente compatible con MCP al servidor compilado, por ejemplo en el
claude_desktop_config.json de Claude Desktop:
{
"mcpServers": {
"qa-copilot": {
"command": "node",
"args": ["/absolute/path/to/qa-copilot-mcp-server/dist/index.js"]
}
}
}Define la variable de entorno GITHUB_TOKEN para elevar el límite de requests sin autenticar de GitHub para get_repo_ci_status (60 peticiones por hora sin él; 5,000 por hora con él) — funciona bien sin token para un uso ocasional.
Cada push y pull request sobre main ejecuta un único job: type-check
(tsc --noEmit), la suite de tests unitarios (vitest run), un build de producción y un
smoke test de protocolo (node smoke-test.mjs) contra el build generado. No se asume ni se requiere acceso a red en ningún punto de CI: get_repo_ci_status solo se ejerce contra FakeGitHubClient en los tests, y el smoke test acepta un resultado de get_repo_ci_status sin red como válido siempre que el servidor lo notifique correctamente por el canal isError de MCP en lugar de estrellarse.
No es un dashboard genérico de GitHub ni de CI, sino un conjunto pequeño y enfocado de herramientas construidas específicamente alrededor de las formas de Playwright-report y GitHub-Actions que de verdad producen los otros repos de este portfolio; está pensado para mostrar cómo un asistente de IA se conecta con datos de prueba reales y verificables en vez de que se le pida razonarlo sobre resultados de tests a partir de un paráfrasis.
Given a Playwright (or similar) error message, returns a heuristic, pattern-matched guess at the root cause and a suggested next step. Fully offline - no LLM call.
| Name | Required | Description | Default |
|---|---|---|---|
| errorMessage | Yes | The raw error/assertion message from a failed test. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since there are no annotations, the description carries the full burden of behavioral disclosure. It does well by stating that the output is a 'guess', that the analysis is 'heuristic, pattern-matched', and that it is 'Fully offline - no LLM call'. This conveys uncertainty and operational constraints clearly, though a bit more detail about output format or confidence would have made it even richer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences accomplish all necessary tasks: defining input, action, output, and an important operational constraint. The key behavioral details are front-loaded, with no wasted or repetitive wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter, absent annotations, and no output schema, the description is nearly complete: it defines the input, the output type (root cause and next step), and the heuristic/offline behavior. The only minor gap is a precise description of the return value's shape, but for this simple tool the current wording is sufficient for an agent to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema's description of errorMessage already covers 'raw error/assertion message from a failed test'. The tool description adds value by scoping the input to a 'Playwright (or similar) error message', which clarifies the expected domain. With exceptionally high schema coverage at the file level, the description's incremental input is small but relevant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies a specific verb ('returns') and a distinct resource ('a heuristic, pattern-matched guess at the root cause and a suggested next step'). The input is scoped to 'Playwright (or similar) error messages', making it easy to differentiate from sibling tools like summarize_test_run or list_flaky_tests. An agent can understand this tool's purpose without inferring anything.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The opening 'Given a Playwright (or similar) error message' clearly indicates when this tool should be used. However, the description does not explicitly name sibling tools as exclusions or state when not to use it, so the comparison with alternatives is left to the agent's own judgment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Deterministically scaffolds a runnable Playwright test file from a named sequence of data-testid interactions (fill, click, expectText, expectVisible). Template-based, not LLM-based - instant and 100% reproducible.
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| testName | Yes | The test's title, e.g. "logs in with valid credentials". | |
| describeName | No | The test.describe() block title. Defaults to "Generated suite". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description does a solid job of setting expectations: it is template-based, deterministic, instant, reproducible, and produces a runnable file. It does not disclose whether the file is written to disk or returned as text, but the key behavioral traits for agent decision-making are clearly surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, concise and front-loaded. It states the action/resource first, then adds the relevant constraints (deterministic, template-based, reproducible). No sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a code generation tool with three parameters and no output schema, the description communicates enough for most invocation decisions: inputs are structured steps, the output is a runnable Playwright test stub, and generation is deterministic. The main missing piece is whether 'scaffolds' means writing a file, returning code, or both, which could affect how the agent uses or verifies the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents testName and describeName, covering about 67% of parameters. The description adds minor value by characterizing steps as a named sequence of data-testid interactions and enumerating the action types, but it does not provide additional meaning about testName or describeName beyond the schema. The nested schema definition of steps already covers most of the needed detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('scaffolds') with a clear resource ('a runnable Playwright test file') and defines the input as a sequence of data-testid interactions with named action types. It also differentiates itself from analysis-focused siblings like summarize_test_run and explain_failure by emphasizing instant, deterministic generation rather than interpretation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used to generate a Playwright test stub when you have a deterministic sequence of data-testid interactions, and emphasizes that it is not LLM-based, which rules out ad-hoc/creative generation use cases. It does not explicitly name alternatives or exclusion conditions, but no sibling tool competes for the same role.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetches the last N GitHub Actions workflow runs for a public repo (owner/repo) and summarizes pass/fail/in-progress counts alongside the raw run list. Requires network access from wherever this server is running; set GITHUB_TOKEN in the environment to raise the unauthenticated rate limit.
| Name | Required | Description | Default |
|---|---|---|---|
| repo | Yes | Repo name, e.g. "healthcare-qa-automation-framework". | |
| limit | No | How many recent runs to fetch (default 10). | |
| owner | Yes | Repo owner, e.g. "sharath9271-design". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description bears the full burden, and it adds real value: it reveals that the tool needs network access, only works on public repos, and can raise its rate limit by setting GITHUB_TOKEN. 'Fetches' implies a read-only operation, and the description does not conceal any dangerous or surprising side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core fetching and summarizing behavior, followed by concise operational caveats. Nothing feels like filler; both sentences earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, three-parameter tool with no output schema, the description covers everything essential: what is fetched, what the response contains, public-repo scope, network requirements, and the optional auth token. An agent has enough context to decide whether to reply and how to handle it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all three parameters (owner, repo, limit) with types, explanations, and examples, covering 100% of its parameters. The description merely restates the limit in prose as 'last N', so it does not add meaningful parameter semantics beyond what the schema conveys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb-resource pair: 'Fetches the last N GitHub Actions workflow runs' and 'summarizes pass/fail/in-progress counts alongside the raw run list'. This clearly distinguishes it from the sibling test-analysis tools, which talk about tests, flakes, and failures rather than repo-level CI status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied and fairly obvious: an agent would call it when asked for recent CI status of a public repo. However, it never names a sibling alternative or states when this tool is not appropriate, so routing is left to inference rather than explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compares the same tests across two or more Playwright JSON reports (e.g. the last N CI runs of the same suite) and returns every test whose outcome wasn't consistent - it passed in some runs and failed/timed out in others.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJsons | Yes | Two or more Playwright JSON reporter outputs, oldest first, as raw strings. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description correctly conveys that this is a read-only comparison and what task qualifies as flaky. However, it does not disclose how missing tests across reports are treated, whether the result is influenced by report ordering beyond the schema note, or what fields are returned per test — leaving some behavioral details implied rather than explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence that states the operation, the input shape, the example use, and the result. There is no filler, and the purpose appears immediately. It is concise despite the long sentence because every clause adds new information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter query tool with no output schema, the description explains the input to the comparison, what counts as 'flaky', and what the output represents. It omits edge-case details, such as how test records without entries in all reports are handled or the exact shape of each returned test, but the overall information is sufficient for an agent to invoke it correctly for a typical flaky-test scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents reportJsons as raw Playwright JSON reporter outputs, oldest first, with minItems=2 (100% schema coverage). The description adds the meaningful context that these should be 'the last N CI runs of the same suite', which clarifies the intended input beyond the bare schema definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb-resource pair ('Compares the same tests across two or more Playwright JSON reports') and explicitly names the result ('returns every test whose outcome wasn't consistent'). It clearly differentiates this from siblings like get_repo_ci_status or explain_failure by focusing on flaky test detection across multiple runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides contextual use ('two or more... e.g. the last N CI runs of the same suite') and makes the minimum-input condition explicit. It does not spell out 'when not to use' or name alternatives, but the multi-report scope plus sibling tool context makes the boundary reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Parses Playwright's JSON reporter output and returns pass/fail/timeout/skipped/flaky counts, total duration, and full details on every failure.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJson | Yes | The raw contents of a Playwright JSON reporter output file (results.json). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full burden; it handles this well by describing what the tool does: parse JSON and return counts, duration, and failure details. It doesn't discuss edge cases, output formatting, or potential errors, but for a straightforward parsing and summarization tool this is a substantially clear behavioral explanation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately states the verb, the input, and the exact outputs. It is front-loaded with 'Parses...' and contains no filler or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, conceptually simple tool, the description fully covers what the input is (Playwright JSON reporter output), what the tool produces, and how the result will be structured. No output schema is present, but the return values are described in enough detail for an agent to consume the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter is already fully documented—'The raw contents of a Playwright JSON reporter output file (results.json)'. The tool description adds no additional parameter-level meaning beyond re-stating that it parses Playwright JSON, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear action ('parses Playwright's JSON reporter output') and reports a specific set of results: pass/fail/timeout/skipped/flaky counts, total duration, and failure details. This distinguishes it from sibling tools like list_flaky_tests or explain_failure, which have narrower or different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended context is clear: this tool is used when a Playwright JSON reporter output is available and a summary of counts, duration, and failures is needed. It does not explicitly mention when not to use it or compare itself with sibling tools, but the usage context is strongly implied and unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0explain_failuregenerate_playwright_test_stubget_repo_ci_statuslist_flaky_testssummarize_test_runScored across 5 tools
Each tool targets a distinct responsibility—summarizing test output, identifying flaky tests, explaining failures, generating test stubs, and fetching CI status. There is no meaningful overlap between any pair of tools.
All tool names follow a consistent verb_noun pattern with clear action-prefixed names: summarize_, list_, explain_, generate_, get_. Minor pluralization differences do not create confusion.
Five tools cover a tightly scoped QA workflow—analyze results, detect flakiness, explain failures, generate tests, and check CI—without redundancy or bloat.
The core QA reporting and triage loop is well supported, from parsing reports to explaining failures and detecting flaky tests. Minor gaps like actionable test-edit or test-deletion operations are absent, but they fall outside the visible purpose.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
Create and manage test cases, test runs and results in Testiny from your AI assistant.
