Model Context Protocol を実装したサーバーです。AI アシスタントに推測をさせるのではなく、Playwright の実行を要約する、複数の実行にわたって flaky テストを検出する、失敗をヒューリスティックに説明する、Playwright テストの雛形を生成する、GitHub リポジトリの最近の CI 状況を取得する、といった具体的な QA／テストインテリジェンスツールの小さなセットを提供します。Claude Desktop、Claude Code、その他の MCP 互換ホストからこのサーバーを指定すれば、それらは自由文の推測ではなく、実際のツール呼び出しになります。
これは、QA 自動化ポートフォリオの中でも「AI に雰囲気ではなく実データを与える」部分に当たります。このポートフォリオ内の他のリポジトリ（healthcare-qa-automation-framework、healthcare-data-quality-framework）は Playwright の JSON レポートと CI 実行を生成しますが、このサーバーはそれらを、アシスタントが実際に照会できるものへと変換します。
プロトコル配線は、素の関数の上に載せる薄い層に過ぎません。 各ツールの実際のロジックは src/playwright/、src/github/、src/tools/ に、小さく（ほぼ）純粋で、それぞれ単体テストされたモジュール群あります。src/index.ts はそれらを MCP SDK に登録するだけです。つまり、間違えにくい部分（レポート解析、flaky 検出、ヒューリスティック）は高速な単体テストでカバーされ、プロトコル配線は別途、エンドツーエンドのスモークテスト（後述）でカバーされます。
GitHub アクセスはプラグ可能です。このポートフォリオ内の他の LLM 統合と同じパターンです。 GitHubClient はインターフェースであり、RealGitHubClient は GitHub REST API への fetch をラップし、FakeGitHubClient は固定データを返します。これにより get_repo_ci_status はオフラインで完全に単体テスト可能になります。一方、実際のクライアントは本物の実行可能なコードです。開発者のマシンで動く MCP ホストには通常のインターネットアクセスがありますが、ネットワークのない環境（このリポジトリ自身の CI サンドボックスなど）でも、そのままテスト可能です。
テストスタブ生成は、意図的に LLM ベースではなく決定論的です。 名前付きの data-testid 操作シーケンスを渡すと、generate_playwright_test_stub は常に同じ出力を生成します。即時・無料・再現可能です。別の仕事のための別のツール（仕様やユーザーストーリーからテスト ケース を生成するもの。そこでは LLM が実際に価値を発揮します）は、このポートフォリオの別プロジェクトになります。
単体テストだけでなく、プロトコルレベルのスモークテストも行います。 smoke-test.mjs はビルドされたサーバーを実際の子プロセスとして起動し、公式 MCP Client を使って stdio 上で操作します。tools/list を実行し、続いてツールごとに tools/call を実行します。これにより CI は、内部の関数だけでなく実際のワイヤプロトコルが機能することを証明します。
Related MCP server: Playwright Debug MCP Server
ツール | 説明 |
| Playwright の JSON レポーター出力を解析し、パス・失敗・タイムアウト・スキップ・ flaky の件数、所要時間、失敗の詳細を返します。 |
| 2 件以上の Playwright JSON レポートを比較し、結果が一貫していないテストを検出します。 |
| 生のエラーメッセージに対して、パターンマッチングによるヒューリスティックな根本原因の推測を返します。完全にオフラインで動作します。 |
| 名前付き |
| 公開 GitHub リポジトリの最近の Actions 実行を取得し、成功・失敗・進行中の件数を要約します。 |
src/
index.ts MCP server: registers all 5 tools, connects over stdio
playwright/reportParser.ts Pure Playwright-JSON parsing: summarizeReport, findFlakyTests
github/githubClient.ts GitHubClient interface + RealGitHubClient (fetch) + FakeGitHubClient
tools/
explainFailure.ts Heuristic pattern-matching (ported from the triage_failure.py
script in healthcare-qa-automation-framework)
generateTestStub.ts Deterministic Playwright test-file template generator
getRepoCiStatus.ts Wraps GitHubClient into pass/fail/in-progress counts
test/ Vitest unit tests for every module above
fixtures/ Sample Playwright JSON reports (3 runs, used for flaky-test tests)
smoke-test.mjs End-to-end MCP protocol smoke test (real child process, real client)
.github/workflows/ci.yml Type-check, unit tests, build, protocol smoke testTypeScript · Node.js · @modelcontextprotocol/sdk · Zod · Vitest · GitHub REST API · GitHub Actions
npm install
npm run build # compiles to dist/
npm test # unit tests (27 tests, fully offline)
npm run smoke # builds, then drives the real server over stdio via the MCP clientビルドしたサーバーを MCP 互換クライアントに指定します。たとえば Claude Desktop の claude_desktop_config.json では:
{
"mcpServers": {
"qa-copilot": {
"command": "node",
"args": ["/absolute/path/to/qa-copilot-mcp-server/dist/index.js"]
}
}
}環境変数 GITHUB_TOKEN を設定すると、実効的に get_repo_ci_status の GitHub レート制限を引き上げられます（トークンなしで 60 リクエスト/時、トークンありで 5,000 リクエスト/時）。たまに使う分には、トークンなしでも問題なく動作します。
main へのすべての push と pull request で、単一のジョブが実行されます。型チェック（tsc --noEmit）、単体テストスイート（vitest run）、プロダクションビルド、そしてビルド成果物に対するプロトコルレベルのスモークテスト（node smoke-test.mjs）です。CI ではどこでもネットワークアクセスを前提とせず、要求もしません。get_repo_ci_status は単体テストで FakeGitHubClient に対してのみ実行され、スモークテストでは、サーバーがクラッシュせず MCP の isError チャネル経由で正常に報告さえすれば、ネットワークなしの get_repo_ci_status の結果は合格として扱います。
これは汎用的な GitHub ダッシュボードや CI ダッシュボードではありません。このポートフォリオの他のリポジトリが実際に生成する Playwright レポートと GitHub Actions の形状に特化した、小さく焦点の絞られたツール群です。AI アシスタントがテスト結果を要約情報へ言い換えるのではなく、実際の検証可能なテストデータに接続されることを示すためのものです。
Given a Playwright (or similar) error message, returns a heuristic, pattern-matched guess at the root cause and a suggested next step. Fully offline - no LLM call.
| Name | Required | Description | Default |
|---|---|---|---|
| errorMessage | Yes | The raw error/assertion message from a failed test. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since there are no annotations, the description carries the full burden of behavioral disclosure. It does well by stating that the output is a 'guess', that the analysis is 'heuristic, pattern-matched', and that it is 'Fully offline - no LLM call'. This conveys uncertainty and operational constraints clearly, though a bit more detail about output format or confidence would have made it even richer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences accomplish all necessary tasks: defining input, action, output, and an important operational constraint. The key behavioral details are front-loaded, with no wasted or repetitive wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter, absent annotations, and no output schema, the description is nearly complete: it defines the input, the output type (root cause and next step), and the heuristic/offline behavior. The only minor gap is a precise description of the return value's shape, but for this simple tool the current wording is sufficient for an agent to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema's description of errorMessage already covers 'raw error/assertion message from a failed test'. The tool description adds value by scoping the input to a 'Playwright (or similar) error message', which clarifies the expected domain. With exceptionally high schema coverage at the file level, the description's incremental input is small but relevant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies a specific verb ('returns') and a distinct resource ('a heuristic, pattern-matched guess at the root cause and a suggested next step'). The input is scoped to 'Playwright (or similar) error messages', making it easy to differentiate from sibling tools like summarize_test_run or list_flaky_tests. An agent can understand this tool's purpose without inferring anything.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The opening 'Given a Playwright (or similar) error message' clearly indicates when this tool should be used. However, the description does not explicitly name sibling tools as exclusions or state when not to use it, so the comparison with alternatives is left to the agent's own judgment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Deterministically scaffolds a runnable Playwright test file from a named sequence of data-testid interactions (fill, click, expectText, expectVisible). Template-based, not LLM-based - instant and 100% reproducible.
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| testName | Yes | The test's title, e.g. "logs in with valid credentials". | |
| describeName | No | The test.describe() block title. Defaults to "Generated suite". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description does a solid job of setting expectations: it is template-based, deterministic, instant, reproducible, and produces a runnable file. It does not disclose whether the file is written to disk or returned as text, but the key behavioral traits for agent decision-making are clearly surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, concise and front-loaded. It states the action/resource first, then adds the relevant constraints (deterministic, template-based, reproducible). No sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a code generation tool with three parameters and no output schema, the description communicates enough for most invocation decisions: inputs are structured steps, the output is a runnable Playwright test stub, and generation is deterministic. The main missing piece is whether 'scaffolds' means writing a file, returning code, or both, which could affect how the agent uses or verifies the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents testName and describeName, covering about 67% of parameters. The description adds minor value by characterizing steps as a named sequence of data-testid interactions and enumerating the action types, but it does not provide additional meaning about testName or describeName beyond the schema. The nested schema definition of steps already covers most of the needed detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('scaffolds') with a clear resource ('a runnable Playwright test file') and defines the input as a sequence of data-testid interactions with named action types. It also differentiates itself from analysis-focused siblings like summarize_test_run and explain_failure by emphasizing instant, deterministic generation rather than interpretation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used to generate a Playwright test stub when you have a deterministic sequence of data-testid interactions, and emphasizes that it is not LLM-based, which rules out ad-hoc/creative generation use cases. It does not explicitly name alternatives or exclusion conditions, but no sibling tool competes for the same role.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetches the last N GitHub Actions workflow runs for a public repo (owner/repo) and summarizes pass/fail/in-progress counts alongside the raw run list. Requires network access from wherever this server is running; set GITHUB_TOKEN in the environment to raise the unauthenticated rate limit.
| Name | Required | Description | Default |
|---|---|---|---|
| repo | Yes | Repo name, e.g. "healthcare-qa-automation-framework". | |
| limit | No | How many recent runs to fetch (default 10). | |
| owner | Yes | Repo owner, e.g. "sharath9271-design". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description bears the full burden, and it adds real value: it reveals that the tool needs network access, only works on public repos, and can raise its rate limit by setting GITHUB_TOKEN. 'Fetches' implies a read-only operation, and the description does not conceal any dangerous or surprising side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core fetching and summarizing behavior, followed by concise operational caveats. Nothing feels like filler; both sentences earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, three-parameter tool with no output schema, the description covers everything essential: what is fetched, what the response contains, public-repo scope, network requirements, and the optional auth token. An agent has enough context to decide whether to reply and how to handle it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all three parameters (owner, repo, limit) with types, explanations, and examples, covering 100% of its parameters. The description merely restates the limit in prose as 'last N', so it does not add meaningful parameter semantics beyond what the schema conveys. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb-resource pair: 'Fetches the last N GitHub Actions workflow runs' and 'summarizes pass/fail/in-progress counts alongside the raw run list'. This clearly distinguishes it from the sibling test-analysis tools, which talk about tests, flakes, and failures rather than repo-level CI status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied and fairly obvious: an agent would call it when asked for recent CI status of a public repo. However, it never names a sibling alternative or states when this tool is not appropriate, so routing is left to inference rather than explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compares the same tests across two or more Playwright JSON reports (e.g. the last N CI runs of the same suite) and returns every test whose outcome wasn't consistent - it passed in some runs and failed/timed out in others.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJsons | Yes | Two or more Playwright JSON reporter outputs, oldest first, as raw strings. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description correctly conveys that this is a read-only comparison and what task qualifies as flaky. However, it does not disclose how missing tests across reports are treated, whether the result is influenced by report ordering beyond the schema note, or what fields are returned per test — leaving some behavioral details implied rather than explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence that states the operation, the input shape, the example use, and the result. There is no filler, and the purpose appears immediately. It is concise despite the long sentence because every clause adds new information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter query tool with no output schema, the description explains the input to the comparison, what counts as 'flaky', and what the output represents. It omits edge-case details, such as how test records without entries in all reports are handled or the exact shape of each returned test, but the overall information is sufficient for an agent to invoke it correctly for a typical flaky-test scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents reportJsons as raw Playwright JSON reporter outputs, oldest first, with minItems=2 (100% schema coverage). The description adds the meaningful context that these should be 'the last N CI runs of the same suite', which clarifies the intended input beyond the bare schema definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb-resource pair ('Compares the same tests across two or more Playwright JSON reports') and explicitly names the result ('returns every test whose outcome wasn't consistent'). It clearly differentiates this from siblings like get_repo_ci_status or explain_failure by focusing on flaky test detection across multiple runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides contextual use ('two or more... e.g. the last N CI runs of the same suite') and makes the minimum-input condition explicit. It does not spell out 'when not to use' or name alternatives, but the multi-report scope plus sibling tool context makes the boundary reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Parses Playwright's JSON reporter output and returns pass/fail/timeout/skipped/flaky counts, total duration, and full details on every failure.
| Name | Required | Description | Default |
|---|---|---|---|
| reportJson | Yes | The raw contents of a Playwright JSON reporter output file (results.json). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full burden; it handles this well by describing what the tool does: parse JSON and return counts, duration, and failure details. It doesn't discuss edge cases, output formatting, or potential errors, but for a straightforward parsing and summarization tool this is a substantially clear behavioral explanation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately states the verb, the input, and the exact outputs. It is front-loaded with 'Parses...' and contains no filler or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, conceptually simple tool, the description fully covers what the input is (Playwright JSON reporter output), what the tool produces, and how the result will be structured. No output schema is present, but the return values are described in enough detail for an agent to consume the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter is already fully documented—'The raw contents of a Playwright JSON reporter output file (results.json)'. The tool description adds no additional parameter-level meaning beyond re-stating that it parses Playwright JSON, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear action ('parses Playwright's JSON reporter output') and reports a specific set of results: pass/fail/timeout/skipped/flaky counts, total duration, and failure details. This distinguishes it from sibling tools like list_flaky_tests or explain_failure, which have narrower or different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended context is clear: this tool is used when a Playwright JSON reporter output is available and a summary of counts, duration, and failures is needed. It does not explicitly mention when not to use it or compare itself with sibling tools, but the usage context is strongly implied and unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0explain_failuregenerate_playwright_test_stubget_repo_ci_statuslist_flaky_testssummarize_test_runScored across 5 tools
Each tool targets a distinct responsibility—summarizing test output, identifying flaky tests, explaining failures, generating test stubs, and fetching CI status. There is no meaningful overlap between any pair of tools.
All tool names follow a consistent verb_noun pattern with clear action-prefixed names: summarize_, list_, explain_, generate_, get_. Minor pluralization differences do not create confusion.
Five tools cover a tightly scoped QA workflow—analyze results, detect flakiness, explain failures, generate tests, and check CI—without redundancy or bloat.
The core QA reporting and triage loop is well supported, from parsing reports to explaining failures and detecting flaky tests. Minor gaps like actionable test-edit or test-deletion operations are absent, but they fall outside the visible purpose.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers. The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites: HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps. Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client. Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
Create and manage test cases, test runs and results in Testiny from your AI assistant.
