playwright-kiwi-sync

license: MIT · node: >=18 · Model Context Protocol: server · tests: 20 e2e checks

An MCP server that syncs automated test results — Playwright, Allure, or any JUnit XML (Maestro, Cypress, pytest…) — straight into existing Kiwi TCMS test executions. No more "mark case 4001 as passed" one at a time in chat.

It's a companion to the team's kiwi-tcms-mcp, not a replacement: that one is for everything manual (browsing plans, creating cases, ad-hoc lookups); this one runs after a test job and records what happened.

• Reads Playwright's json report, Allure's raw allure-results/, or a JUnit XML file — and matches each test to a Kiwi case via a [C1234] tag or annotation

• Writes the execution status (PASSED/FAILED/BLOCKED) into the run, adds the failure message as a comment, uploads screenshot/trace attachments

• Fast: one read for the whole run, then bounded-parallel writes — a 300-test suite is ~1 read + 300 writes (5 in flight), not 600 sequential calls

• Safe by default: dryRun shows exactly what would change; cases missing from the run are reported, not silently added (opt in with addMissingCases)

• Correct by construction: execution statuses are resolved by name from your instance (TestExecutionStatus.filter), never hardcoded ids; several results on one case (data-driven tests) collapse to the worst outcome, deterministically

• Guarded escape hatch: kiwi_rpc refuses *.remove({}) outright and needs an explicit allowDestructive for any delete; oversized results are capped so one broad filter can't flood the model's context

• Runs locally with your own Kiwi login — it can only do what your account can

Playwright json / allure-results/ / junit.xml
                    |
MCP client  <--stdio-->  playwright-kiwi-sync (Node)  <--HTTPS JSON-RPC-->  Kiwi TCMS

Tools

All three sync tools take runId, dryRun and addMissingCases.

A note on report formats

This reads raw, structured result data. It does not parse rendered HTML reports (Playwright's or Allure's) — those are generated static sites with no stable format meant to be read back, and scraping them would break on any reporter version bump. Keep generating HTML for humans; just also emit json/junit (Playwright) or keep allure-results/ around (Allure). They run side by side.

Related MCP server: testit-mcp-server

How it compares

The official plugin is the right tool if you want Kiwi to generate cases from a CI run. This is the right tool if you already have curated cases in Kiwi (e.g. migrated from TestRail) and want automation results recorded against those.

Tagging tests for Kiwi

A [C1234] tag in the test name works for every format. Annotation/label forms keep titles clean but are reporter-specific:

// Works everywhere (Playwright json, Allure, JUnit, Maestro flow names)
test("[C4001] rejects an expired token", async ({ page }) => { ... });

// Playwright json reporter
test("rejects an expired token", async ({}, testInfo) => {
  testInfo.annotations.push({ type: "kiwi-case", description: "4001" });
});

// Allure
import { allure } from "allure-js-commons";
test("rejects an expired token", async () => { allure.label("kiwi-case", "4001"); });

Setup

New here? Follow SETUP.md — step-by-step, cross-platform.

git clone https://github.com/ZainAbdeen/playwright-kiwi-sync
cd playwright-kiwi-sync
npm install
cp .env.example .env   # your own Kiwi login
npm test               # smoke + 20 end-to-end checks against a built-in mock Kiwi

Then add a kiwi-sync block under mcpServers in ~/.claude.json, next to your existing kiwi block — full JSON in SETUP.md.

Example prompts

"Dry-run sync results.json into Kiwi run 87."

"Sync results.json into run 87, add any cases that aren't in it yet."

"Sync the Maestro junit.xml into run 92."

"What's the status of regression run 87?"

Testing

npm run e2e boots an in-memory fake of Kiwi's JSON-RPC endpoint (scripts/mock-kiwi.js), starts the real MCP server against it, and drives every tool through a real MCP client. It covers: dry-run writes nothing, correct executions updated, failure comments, missing-case reporting vs addMissingCases, JUnit attribute decoding, run summary, kiwi_rpc passthrough, and the -32098 permission-denied path (surfaced per case with a hint, and not mistaken for an expired session), the escape-hatch guards (empty-filter delete refused, explicit delete needs allowDestructive, oversized output truncated), and a dead-tunnel HTML/502 response diagnosed plainly instead of crashing on JSON parse.

Method names and the values shape were checked against the Kiwi 16.4 API docs and source (TestExecution.update binds a ModelForm whose field is status, not status_id; add_comment, add_attachment (since 15.3), add_link and TestRun.add_case all exist). What the mock can't prove is your instance's permissions and any local customisation — hence dry-run first, see SETUP.md.

Known gaps

• Needs a Kiwi account that can update executions. A create-only account (the -32098 case) will get a clear per-case error, but nothing will sync.

• No CI/webhook trigger yet — invoked from a Claude prompt. A CI step calling sync_junit_run after the test job is the natural next step.

• Several results on one case collapse to the worst status (FAILED/ERROR > BLOCKED > PASSED) with all failure messages kept — there's no per-variant record in Kiwi.

• JUnit has no attachment element, so sync_junit_run records status + failure message only.

License

MIT