一个 Model Context Protocol 服务器，它为 AI 助手提供一组小而具体的 QA/测试智能工具，而不是让它凭空猜测：汇总一次 Playwright 运行、在多次运行中找出不稳定（flaky）的测试、以启发式的方式解释失败原因、搭建一个 Playwright 测试脚手架，以及读取 GitHub 仓库近期的 CI 状态。把它指向 Claude Desktop、Claude Code 或任何其他兼容 MCP 的主机，这些能力就会变成真正的工具调用，而不是自由文本式的猜测。
这是 QA 自动化作品集里“给 AI 真实数据，而不是凭感觉”的那一块：该作品集里的其他仓库（healthcare-qa-automation-framework、healthcare-data-quality-framework）负责产出 Playwright JSON 报告和 CI 运行记录；而这个服务器就是把这些数据变成助手真正可以查询的内容。
协议接线只是覆盖在普通函数之上的一层薄壳。 每个工具的真实逻辑都放在 src/playwright/、src/github/ 和 src/tools/ 中，作为可独立单测的、小型的（近似）纯函数模块存在。src/index.ts 只负责向 MCP SDK 注册它们——所以最容易被搞错的部分（报告解析、flaky 检测、启发式分析）由快速的单元测试覆盖，而协议接线则由独立的端到端冒烟测试覆盖（见下文）。
GitHub 访问采用可插拔设计，这与本作品中其他 LLM 集成的模式相同。 GitHubClient 是一个接口；RealGitHubClient 在 fetch 外层封装了 GitHub REST API，而 FakeGitHubClient 返回预设数据。这让 get_repo_ci_status 可以在完全离线的环境下做完整的单元测试，而真实客户端同时又是可运行的真实代码——运行在开发者机器上的 MCP 主机拥有正常的互联网访问权限，即使在某些没有网络的环境（比如本仓库自己的 CI 沙箱）中也是如此。
测试脚手架生成是确定性的，而不是基于 LLM，这是有意的选择。 给定一组具名的 data-testid 交互序列，generate_playwright_test_stub 总是生成相同的输出——即时、免费、可复现。另一类工具负责另一类任务（根据规格说明或用户故事编写测试用例，那种场景下 LLM 才真正创造价值），那是本作品集的另一个项目。
不仅有单元测试，还有协议层面的冒烟测试。 smoke-test.mjs 将构建好的服务器作为真正的子进程启动，并通过 stdio 用官方 MCP Client 驱动它——先 tools/list，再对每个工具调用一次 tools/call——因此 CI 证明的是真实线上协议可工作，而不只是其背后的函数可用。
Related MCP server: Playwright Debug MCP Server
工具 | 说明 |
| 解析 Playwright JSON reporter 的输出，统计通过/失败/超时/跳过/flaky 次数、运行时长，并给出完整的失败详情。 |
| 对比 2 份及以上 Playwright JSON 报告中的相同测试，并标记结果不一致的测试。 |
| 对原始报错信息做基于启发式、模式匹配的根因猜测，完全离线可用。 |
| 根据一组合法的 |
| 在 GitHub 上读取一个公开仓库近期的 Action 运行情况，汇总通过/失败/进行中的次数。 |
src/
index.ts MCP server: registers all 5 tools, connects over stdio
playwright/reportParser.ts Pure Playwright-JSON parsing: summarizeReport, findFlakyTests
github/githubClient.ts GitHubClient interface + RealGitHubClient (fetch) + FakeGitHubClient
tools/
explainFailure.ts Heuristic pattern-matching (ported from the triage_failure.py
script in healthcare-qa-automation-framework)
generateTestStub.ts Deterministic Playwright test-file template generator
getRepoCiStatus.ts Wraps GitHubClient into pass/fail/in-progress counts
test/ Vitest unit tests for every module above
fixtures/ Sample Playwright JSON reports (3 runs, used for flaky-test tests)
smoke-test.mjs End-to-end MCP protocol smoke test (real child process, real client)
.github/workflows/ci.yml Type-check, unit tests, build, protocol smoke testTypeScript · Node.js · @modelcontextprotocol/sdk · Zod · Vitest · GitHub REST API · GitHub Actions
npm install
npm run build # compiles to dist/
npm test # unit tests (27 tests, fully offline)
npm run smoke # builds, then drives the real server over stdio via the MCP client把任意一个兼容 MCP 的客户端指向构建后的服务器，例如在 Claude Desktop 的 claude_desktop_config.json 里：
{
"mcpServers": {
"qa-copilot": {
"command": "node",
"args": ["/absolute/path/to/qa-copilot-mcp-server/dist/index.js"]
}
}
}设置环境变量 GITHUB_TOKEN 可以提升 get_repo_ci_status 在未登录 GitHub 状态下的速率限制（不设置时每小时 60 次请求，有 token 时每小时 5,000 次）——不配置 token 也完全可行用于程序，只是适合偶发使用。
每次针对 main 的 push 和拉取请求都会运行一个作业：类型检查（tsc --noEmit）、单元测试套件（vitest run）、生产构建、以及针对构建产物的协议级冒烟测试（node smoke-test.mjs）。CI 全程不假设也不需要网络：get_repo_ci_status 在单元测试中只对 FakeGitHubClient 进行验证；冒烟测试则把无网络状态下的 get_repo_ci_status 结果视为通过，只要服务器通过 MCP 的 isError 通道干净地报告结果而不是崩溃。
这不是一个通用的 GitHub或 CI 仪表盘——它是一组小巧、聚焦的工具，专门围绕该作品集中其他仓库实际产生的 Playwright 测试报告和 GitHub Actions 数据形态来构建，目的是演示如何让 AI 助手接入真实、可验证的测试数据，而不是让它根据一段转述去推断测试结果。
Given a Playwright (or similar) error message, returns a heuristic, pattern-matched guess at the root cause and a suggested next step. Fully offline - no LLM call.
| Name | Required | Description | Default |
|---|---|---|---|
| errorMessage | Yes | The raw error/assertion message from a failed test. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since there are no annotations, the description carries the full burden of behavioral disclosure. It does well by stating that the output is a 'guess', that the analysis is 'heuristic, pattern-matched', and that it is 'Fully offline - no LLM call'. This conveys uncertainty and operational constraints clearly, though a bit more detail about output format or confidence would have made it even richer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences accomplish all necessary tasks: defining input, action, output, and an important operational constraint. The key behavioral details are front-loaded, with no wasted or repetitive wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter, absent annotations, and no output schema, the description is nearly complete: it defines the input, the output type (root cause and next step), and the heuristic/offline behavior. The only minor gap is a precise description of the return value's shape, but for this simple tool the current wording is sufficient for an agent to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema's description of errorMessage already covers 'raw error/assertion message from a failed test'. The tool description adds value by scoping the input to a 'Playwright (or similar) error message', which clarifies the expected domain. With exceptionally high schema coverage at the file level, the description's incremental input is small but relevant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies a specific verb ('returns') and a distinct resource ('a heuristic, pattern-matched guess at the root cause and a suggested next step'). The input is scoped to 'Playwright (or similar) error messages', making it easy to differentiate from sibling tools like summarize_test_run or list_flaky_tests. An agent can understand this tool's purpose without inferring anything.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The opening 'Given a Playwright (or similar) error message' clearly indicates when this tool should be used. However, the description does not explicitly name sibling tools as exclusions or state when not to use it, so the comparison with alternatives is left to the agent's own judgment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Deterministically scaffolds a runnable Playwright test file from a named sequence of data-testid interactions (fill, click, expectText, expectVisible). Template-based, not LLM-based - instant and 100% reproducible.
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| testName | Yes | The test's title, e.g. "logs in with valid credentials". | |
| describeName | No | The test.describe() block title. Defaults to "Generated suite". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description does a solid job of setting expectations: it is template-based, deterministic, instant, reproducible, and produces a runnable file. It does not disclose whether the file is written to disk or returned as text, but the key behavioral traits for agent decision-making are clearly surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, concise and front-loaded. It states the action/resource first, then adds the relevant constraints (deterministic, template-based, reproducible). No sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a code generation tool with three parameters and no output schema, the description communicates enough for most invocation decisions: inputs are structured steps, the output is a runnable Playwright test stub, and generation is deterministic. The main missing piece is whether 'scaffolds' means writing a file, returning code, or both, which could affect how the agent uses or verifies the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents testName and describeName, covering about 67% of parameters. The description adds minor value by characterizing steps as a named sequence of data-testid interactions and enumerating the action types, but it does not provide additional meaning about testName or describeName beyond the schema. The nested schema definition of steps already covers most of the needed detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('scaffolds') with a clear resource ('a runnable Playwright test file') and defines the input as a sequence of data-testid interactions with named action types. It also differentiates itself from analysis-focused siblings like summarize_test_run and explain_failure by emphasizing instant, deterministic generation rather than interpretation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used to generate a Playwright test stub when you have a deterministic sequence of data-testid interactions, and emphasizes that it is not LLM-based, which rules out ad-hoc/creative generation use cases. It does not explicitly name alternatives or exclusion conditions, but no sibling tool competes for the same role.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetches the last N GitHub Actions workflow runs for a public repo (owner/repo) and summarizes pass/fail/in-progress counts alongside the raw run list. Requires network access from wherever this server is running; set GITHUB_TOKEN in the environment to raise the unauthenticated rate limit.
| Name | Required | Description | Default |
|---|---|---|---|
| repo | Yes | Repo name, e.g. "healthcare-qa-automation-framework". | |
| limit | No | How many recent runs to fetch (default 10). | |
| owner | Yes | Repo owner, e.g. "sharath9271-design". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description bears the full burden, and it adds real value: it reveals that the tool needs network access, only works on public repos, and can raise its rate limit by setting GITHUB_TOKEN. 'Fetches' implies a read-only operation, and the description does not conceal any dangerous or surprising side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core fetching and summarizing behavior, followed by concise operational caveats. Nothing feels like filler; both sentences earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, three-parameter tool with no output schema, the description covers everything essential: what is fetched, what the response contains, public-repo scope, network requirements, and the optional auth token. An agent has enough context to decide whether to reply and how to handle it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all three parameters (owner, repo, limit) with types, explanations, and examples, covering 100% of its parameters. The description merely restates the limit in prose as 'last N', so it does not add meaningful parameter semantics beyond what the schema conveys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb-resource pair: 'Fetches the last N GitHub Actions workflow runs' and 'summarizes pass/fail/in-progress counts alongside the raw run list'. This clearly distinguishes it from the sibling test-analysis tools, which talk about tests, flakes, and failures rather than repo-level CI status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied and fairly obvious: an agent would call it when asked for recent CI status of a public repo. However, it never names a sibling alternative or states when this tool is not appropriate, so routing is left to inference rather than explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compares the same tests across two or more Playwright JSON reports (e.g. the last N CI runs of the same suite) and returns every test whose outcome wasn't consistent - it passed in some runs and failed/timed out in others.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJsons | Yes | Two or more Playwright JSON reporter outputs, oldest first, as raw strings. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description correctly conveys that this is a read-only comparison and what task qualifies as flaky. However, it does not disclose how missing tests across reports are treated, whether the result is influenced by report ordering beyond the schema note, or what fields are returned per test — leaving some behavioral details implied rather than explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence that states the operation, the input shape, the example use, and the result. There is no filler, and the purpose appears immediately. It is concise despite the long sentence because every clause adds new information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter query tool with no output schema, the description explains the input to the comparison, what counts as 'flaky', and what the output represents. It omits edge-case details, such as how test records without entries in all reports are handled or the exact shape of each returned test, but the overall information is sufficient for an agent to invoke it correctly for a typical flaky-test scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents reportJsons as raw Playwright JSON reporter outputs, oldest first, with minItems=2 (100% schema coverage). The description adds the meaningful context that these should be 'the last N CI runs of the same suite', which clarifies the intended input beyond the bare schema definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb-resource pair ('Compares the same tests across two or more Playwright JSON reports') and explicitly names the result ('returns every test whose outcome wasn't consistent'). It clearly differentiates this from siblings like get_repo_ci_status or explain_failure by focusing on flaky test detection across multiple runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides contextual use ('two or more... e.g. the last N CI runs of the same suite') and makes the minimum-input condition explicit. It does not spell out 'when not to use' or name alternatives, but the multi-report scope plus sibling tool context makes the boundary reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Parses Playwright's JSON reporter output and returns pass/fail/timeout/skipped/flaky counts, total duration, and full details on every failure.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJson | Yes | The raw contents of a Playwright JSON reporter output file (results.json). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full burden; it handles this well by describing what the tool does: parse JSON and return counts, duration, and failure details. It doesn't discuss edge cases, output formatting, or potential errors, but for a straightforward parsing and summarization tool this is a substantially clear behavioral explanation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately states the verb, the input, and the exact outputs. It is front-loaded with 'Parses...' and contains no filler or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, conceptually simple tool, the description fully covers what the input is (Playwright JSON reporter output), what the tool produces, and how the result will be structured. No output schema is present, but the return values are described in enough detail for an agent to consume the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter is already fully documented—'The raw contents of a Playwright JSON reporter output file (results.json)'. The tool description adds no additional parameter-level meaning beyond re-stating that it parses Playwright JSON, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear action ('parses Playwright's JSON reporter output') and reports a specific set of results: pass/fail/timeout/skipped/flaky counts, total duration, and failure details. This distinguishes it from sibling tools like list_flaky_tests or explain_failure, which have narrower or different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended context is clear: this tool is used when a Playwright JSON reporter output is available and a summary of counts, duration, and failures is needed. It does not explicitly mention when not to use it or compare itself with sibling tools, but the usage context is strongly implied and unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0explain_failuregenerate_playwright_test_stubget_repo_ci_statuslist_flaky_testssummarize_test_runScored across 5 tools
Each tool targets a distinct responsibility—summarizing test output, identifying flaky tests, explaining failures, generating test stubs, and fetching CI status. There is no meaningful overlap between any pair of tools.
All tool names follow a consistent verb_noun pattern with clear action-prefixed names: summarize_, list_, explain_, generate_, get_. Minor pluralization differences do not create confusion.
Five tools cover a tightly scoped QA workflow—analyze results, detect flakiness, explain failures, generate tests, and check CI—without redundancy or bloat.
The core QA reporting and triage loop is well supported, from parsing reports to explaining failures and detecting flaky tests. Minor gaps like actionable test-edit or test-deletion operations are absent, but they fall outside the visible purpose.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
Create and manage test cases, test runs and results in Testiny from your AI assistant.
