Muggle Works

Acceptance test agent for coding agents: real-browser tests, with screenshots on the PR.

Coding agents are good at producing a diff. They are bad at knowing whether the diff works, whether it broke the login flow, and what still has to happen before a human can merge it. Muggle Works is the harness around the agent that answers those questions: it freezes requirements, delegates the design and build, runs the unit suite, drives a real browser through the affected user flows, opens the pull request with screenshots attached, and then keeps watching that PR — picking up review comments, red CI, and a stale base branch until the change is genuinely mergeable.

Works in Claude Code, Cursor, Codex, Windsurf, and any MCP client. Powered by Muggle AI.

AI agents: start with AGENTS.md (install, sign-in, flows, skills, errors) or llms.txt. muggle serve also serves both as MCP resources, muggle://docs/agents and muggle://docs/llms.

Highlights

• One request, one pull request. /mdo "add a logout button" runs requirements → design → build → impact analysis → unit tests → browser acceptance → PR. You answer one questionnaire up front, then walk away.

• A real browser under a real identity. Not a headless stub. Every managed login profile owns a live inbox, so magic links, emailed OTPs, email 2FA, verification mail, and password resets are all testable — no mail catcher, no test-only backdoor, no SKIP_AUTH flag.

• GitHub and GitLab as peers. Pull requests and merge requests, review threads and discussions, Actions and pipelines, signed commits on both. Self-hosted GitLab included — the provider is detected from the remote, not hardcoded.

• The loop keeps running after you close the laptop. A watcher polls the PR and re-enters the cycle when a reviewer comments, CI goes red, or the branch falls behind its base. Feedback becomes commits without you relaying it.

• A gate, not a vibe. No PR is opened until requirements are written, the build typechecks and lints, new logic carries unit tests, the suite passes, and a browser verdict is recorded — or a waiver reason is written down. Silence is not a waiver.

• Plain English in, replayable scripts out. Describe a flow in a sentence; get a script that persists across sessions and re-runs as a regression test after every change.

• 130 MCP tools if you'd rather assemble your own pipeline than use the packaged one.

Built for two kinds of people

If you write code for a living, this is the discipline you'd apply yourself if you had the patience to apply it every time: frozen requirements, a Definition of Done that blocks the PR, evidence attached to the review, and a follow-up loop that doesn't forget.

If you vibe-code, this is the part you can't easily judge by reading the diff. You don't have to know Playwright, or what a fixture is, or why the checkout page broke on mobile. Describe the feature; the harness builds it, clicks through it like a user, shows you the screenshots, and tells you what failed in English.

Related MCP server: Smart Agent Workflow MCP

The cycle

Stage 5.5 refuses to open the PR until build and unit tests meet the Definition of Done, and stage 7 refuses to settle its walkthrough without an E2E verdict. Stage 8 is dispatched only once stage 7.5 clears — a cycle with unrepaired failures never reaches a watcher.

Quick start

1. Install

Claude Code — full plugin: skills, MCP tools, and the browser runner.

/plugin marketplace add multiplex-ai/muggle-ai-works
/plugin install muggleai@muggle-works

The marketplace is muggle-works and its entry is muggleai; the installed plugin is named muggle, so its skills appear as /muggle:<skill>.

Cursor — npm install -g @muggleai/works. The postinstall writes ~/.cursor/mcp.json and syncs the muggle-* skills into ~/.cursor/skills/. Restart Cursor.

Codex — full plugin: skills, MCP tools, and the guardrail hooks.

codex plugin marketplace add multiplex-ai/muggle-ai-works
codex plugin add muggle@muggle-works

Restart Codex and trust the plugin's hooks when it asks (or later, in /hooks). Skills are invoked as $muggle:<skill> — $muggle:muggle-do, $muggle:muggle-test — or picked from /skills. See Codex for what differs.

Windsurf, any other MCP client — install the package as above, then register the server:

{
  "mcpServers": {
    "muggle": {
      "command": "muggle",
      "args": ["serve"],
      "env": {
        "MUGGLE_MCP_PROMPT_SERVICE_TARGET": "production"
      }
    }
  }
}

2. Set up

muggle init

Explains how the harness works, saves your preferences, and offers to install the pull-request walkthrough check. Then confirm the install is healthy:

/mstatus                  # Claude Code
$muggle:muggle-status     # Codex
muggle doctor             # anywhere — also checks the Codex plugin or MCP entry

Broken? /mrepair, or muggle setup --force.

3. Build something

/mdo "Add a logout button to the header"

PRE-FLIGHT    → repo: frontend · branch: users/me/logout-button · target: localhost:3000
REQUIREMENTS  → Goal: logout button in header. AC: visible, ends session, redirects to /login.
BUILD         → src/components/Header.tsx, src/hooks/useLogout.ts (+ unit tests)
IMPACT        → affects "User Login" and "Session" flows
UNIT TESTS    → 12/12 pass
ACCEPTANCE    → 3/3 test cases pass · runId a1b2c3 · 14 screenshots
PULL REQUEST  → #42 opened, walkthrough posted
WATCHER       → armed on #42

Authentication starts on the first protected call: a browser opens with a verification code, you sign in, and the call continues. Credentials persist in ~/.muggle-ai/.

Commands

Every skill has a short alias. Both spellings work in Claude Code. In Codex, mention the skill with $: $muggle:muggle-do, or the alias $muggle:mdo.

GitHub and GitLab

The harness resolves one provider token — github or gitlab — from the URL you passed or the origin remote, then picks the matching recipe set. GitHub goes through gh, GitLab through glab. Self-hosted GitLab resolves by matching the remote host against glab's configured host, so git.acme.com works without configuration.

Nested GitLab namespaces of any depth are handled — the project path is everything before the /-/ segment, never assumed to be two levels.

The pull-request walkthrough check

When a PR is opened from a session running this plugin, Muggle reserves a comment for the acceptance walkthrough and holds the turn open until it's settled. The comment moves through three stages, edited in place: waiting for the E2E feasibility check; then E2E running — waiting for the report or E2E not feasible with the reason; then the E2E report. A PR that changes only docs, tests, lint/format config, or CI config opens straight at its reason. A session that ends without a run or a reason leaves the comment at the stage it reached, and the check stays unsettled.

A PR opened any other way — the web UI, a teammate without the plugin — never passes through that session, so the same check also runs in GitHub Actions. It fails while the comment is still empty, and whenever the walkthrough reports a failed or inconclusive test case; the check's summary names each one with its failing step and a link to the run. While a new E2E run is underway on the PR, the check stays in progress until that run's walkthrough replaces the previous one. When the Muggle PR watcher sees it red, it repairs the cases, re-runs the whole suite, and re-settles the comment.

The autoInstallCiWorkflow preference, chosen during muggle init, decides whether Muggle adds the workflow itself: always installs it in the repository muggle init runs in and in any repository /muggle-do opens a PR from, ask (the default) asks first, never leaves CI alone. To add it to a repository by hand:

muggle ci-install

That writes .github/workflows/muggle-walkthrough.yml. Run again later, it replaces a copy an earlier release wrote with the current one, and leaves a copy you edited alone unless you pass --force; muggle init and /muggle-do do the same refresh on their own unless autoInstallCiWorkflow is never. To add it by hand instead:

name: muggle-walkthrough

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
  issue_comment:
    types: [created, edited]

permissions:
  contents: read
  checks: write
  pull-requests: write

concurrency:
  group: muggle-walkthrough-${{ github.event.pull_request.number || github.event.issue.number }}
  cancel-in-progress: false

jobs:
  walkthrough-comment:
    if: github.event_name == 'pull_request' || github.event.issue.pull_request
    runs-on: ubuntu-latest
    steps:
      - name: Check the walkthrough comment
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: npx -y -p @muggleai/works muggle pr-walkthrough-check --check-run

Four things worth knowing:

• The verdict is published as a check run against the pull request's head commit, not as this job's own status. An issue_comment run executes against the default branch, and only a head-commit check reaches the pull request from there — which is what lets settling the comment turn the check green without a new push.

• GitHub reads the issue_comment trigger from the default branch's copy of the workflow, so comment-driven re-runs start working only once it is merged.

• Runs for one pull request queue rather than cancel. Settling the comment starts a run while the one checking the push may still be going, and cancelling that run would leave a cancelled job on the pull request — red beside a green check. An installed copy that still says cancel-in-progress: true shows exactly that; muggle ci-install updates it.

• On a pull request from a fork, GITHUB_TOKEN is read-only and the check run cannot be published. The command fails open and reports nothing rather than blocking the pull request.

Architecture

Test management lives in the cloud; execution is local and stateless. muggle-remote-* tools own projects, use cases, test cases, and scripts. muggle-local-* tools receive what they need and run the browser — so a local run never waits on cloud replay capacity.

Entity model

Project ("My App")
  └── Use Case ("User Login Flow")
       └── Test Case ("Login with valid credentials")
            └── Test Script (recorded browser steps)
                 └── Run Result (pass/fail + screenshots)

Execution flow

Your agent describes what to test
         │
         ▼
muggle-remote-*  create or find the test cases
         │
         ▼
muggle-local-execute-test-generation  launches the browser runner
         │
         ▼
An agent drives the browser step by step (click, type, navigate, assert)
         │
         ▼
Per-step screenshots → action-script.json recorded
         │
         ▼
Results at ~/.muggle-ai/sessions/{runId}/ — published during the run,
so the result carries a viewUrl straight to the dashboard

Data directory

~/.muggle-ai/
├── oauth-session.json    # OAuth tokens (short-lived, auto-refresh)
├── api-key.json          # Long-lived key for service calls
├── projects/             # Local project cache
├── sessions/             # Run sessions — read one with `muggle run-view [runId]`
│   └── {runId}/
│       ├── action-script.json            # Recorded browser steps
│       ├── results.md                    # Step-by-step report
│       └── electron-runtime/screenshot/  # Per-step images
├── muggle-do/sessions/   # Cycle state, iterations, PR watchers
└── electron-app/{version}/

MCP tool reference

130 tools across authentication, projects, use cases, test cases, test states, scripts, workflows, local execution, reporting, secrets, pricing, billing, and administration. Call them directly from any MCP client to build your own pipeline.

Authentication — muggle-remote-auth-status, -login, -poll, -logout, -register (create an account from an email and password, no browser), plus -api-key-create, -list, -get, -revoke.

Projects — muggle-remote-organization-list names the organizations an organizationId can make a project belong to. muggle-remote-project-create, -list, -get, -update, -delete, plus per-project rollups: -test-results-summary-get, -test-runs-summary-get, -test-scripts-summary-get. Pass primaryResolution, compatibleResolutions and runStrategy (primaryOnly, the default, primaryFirst or allAtOnce) to -create or -update to verify every test case at more than one viewport; muggle-remote-project-display-resolutions-get reads a lane's primary, compatible and enabled resolutions and its run strategy.

Bulk-preview submissions return a jobId immediately. Poll muggle-remote-bulk-preview-job-get until terminal, then persist with the matching -create tool. -list and -cancel round out the set.

Pricing and billing — muggle-remote-pricing-get reads the live plans, token price and token packages without sign-in. muggle-remote-wallet-package-list lists the packages the signed-in user can buy, with the packageId that muggle-remote-wallet-topup and -wallet-auto-topup-update take.

Reporting — muggle-remote-report-stats-summary-get, -cost-query, -final-generate (PDF/HTML/Markdown), -preferences-upsert.

Administration — muggle-remote-prd-file-* (upload, list and delete requirements docs), muggle-remote-secret-* (credentials for test environments, with -secret-group-* and -secret-type-list), muggle-remote-wallet-* (top-ups, payment methods, auto-topup), muggle-remote-recommend-cicd-setup / -recommend-schedule, muggle-remote-user-feedback-*.

Resources — muggle://docs/agents (AGENTS.md) and muggle://docs/llms (llms.txt), read from the installed package through resources/list and resources/read.

CLI reference

# Server
muggle serve                  # Start the MCP server with all tools
muggle serve --e2e            # Cloud tools only (muggle-remote-*)
muggle serve --local          # Local tools only (muggle-local-*)

# Setup and diagnostics
muggle init [--json | --apply <file>]  # Guided setup; saves preferences, offers the CI check
muggle llm-source [--show]    # Who supplies the model for local runs: Muggle AI credit, your Anthropic key, a local model, or your Claude Code
muggle llm-source --use <source> [--base-url <url> --model <name>]  # Set it without prompting
muggle setup [--force]        # Download or update the browser runner
muggle upgrade [--check | --version <v>] [--force]  # Install the latest (or a given) runner version
muggle versions               # List installed runner versions
muggle cleanup [--dry-run] [--all] [--skills]  # Remove old versions and obsolete skills
muggle doctor                 # Diagnose installation problems

# Inspecting runs
muggle run-view [runId]       # A run's steps beside the frames they produced (newest run by default)
muggle run-view --html [--out-dir <dir>]  # Render those frames as a self-contained page
muggle run-view --open        # Render it and open it in the browser
muggle checkout-freshness [--base <branch>] [--no-fetch]  # Fetch the base and check this checkout is built on its latest commit

# Authentication
muggle login [--key-name <name>] [--key-expiry 30d|90d|1y|never]
muggle logout
muggle status

# Pricing
muggle pricing [--json]       # Plans, token price and token packages from the live catalog

# Pull requests
muggle ci-install [--force]   # Add or update the walkthrough check in GitHub Actions
muggle pr-walkthrough-check [--pr <n>] [--check-run]  # Verify a PR's walkthrough comment is settled
muggle build-pr-section [--base <branch>]  # Render a PR evidence block from a report on stdin
muggle e2e-plan [--pr <url>]  # Post the test cases a local run is about to execute (JSON on stdin) to the walkthrough

# Shared local services, between agents in different checkouts
muggle stack claim [--wait <seconds>]  # Claim the ports this checkout's E2E run needs
muggle stack check [--port <port>]     # Confirm the stack still serves this checkout's claim
muggle stack release                   # Give up the lease and place in line
muggle stack status                    # Every lease, the line, and tracked services

# Session and PR-watcher supervision
muggle watch install [--uninstall]     # Schedule the watch with the OS scheduler
muggle watch tick                      # Run one watch pass
muggle watch status                    # Last pass: sessions, processes, PR watches, findings
muggle watch reload-plugins [--dry-run]  # Reload plugins across active Claude sessions
muggle watch register --slot <dir>     # Opt a PR-watch slot into out-of-session recovery
muggle watch unregister --slot <dir>   # Withdraw it

muggle --version
muggle --help

Configuration

Environment targeting. Set MUGGLE_MCP_PROMPT_SERVICE_TARGET (production or dev) in the MCP server's env block — in Codex, export it in your shell; the plugin forwards it. Mismatching it against the account you log in with is the usual cause of unauthorized_client.

Multi-repo cycles. Drop a muggle-repos.json in your working directory so a single request can span services:

[
  { "name": "frontend", "path": "/absolute/path/to/frontend", "testCommand": "pnpm test" },
  { "name": "backend",  "path": "/absolute/path/to/backend",  "testCommand": "pnpm test" }
]

Preferences. /mprefs (or muggle init) controls the gates — whether to use a worktree, rebase onto the base branch, run acceptance tests every cycle, open the PR automatically, and arm the watcher. Each gate takes always, ask, or never.

When installed as a Claude Code plugin, MCP configuration ships with the plugin (plugin/.mcp.json; Codex reads plugin/.codex-mcp.json) — there is nothing to copy by hand.

Client support

Slash commands are plugin-managed; update them with /plugin update muggleai@muggle-works (Claude Code) or codex plugin marketplace upgrade muggle-works (Codex).

Codex

The Codex plugin ships the same skills, MCP server, and hooks as the Claude Code one, with what Codex needs written in rather than left to defaults:

• Long local runs. Codex cancels an MCP call after 60 seconds unless told otherwise, and a local browser run takes minutes. The plugin's server allows 30 minutes per call and two minutes to start.

• Your environment. Codex starts MCP servers with only HOME, PATH and the locale. The plugin forwards what the server reads — MUGGLE_MCP_PROMPT_SERVICE_TARGET, the AUTH0_* and service-URL overrides, HTTPS_PROXY/NO_PROXY, and on Linux the display variables the browser runner needs.

• Claude Code tools in the skills. Codex has no AskUserQuestion, Skill tool, or background monitor. Each skill points at _shared/host-tool-equivalents.md, which maps them to Codex, and the session-start hook names it too.

• PR watching. Codex cannot wake a session when a reviewer comments, so $muggle:muggle-pr-followup handles what is outstanding each time you run it rather than watching in the background.

• Sandbox. Codex's default sandbox blocks the network and writes outside the workspace, so gh/glab and state under ~/.muggle-ai/ ask for approval. To stop the prompts, allow both in ~/.codex/config.toml:

  [sandbox_workspace_write]
  network_access = true
  writable_roots = ["/absolute/path/to/home/.muggle-ai"]

• Headless runs. codex exec cannot ask you anything. It refuses every muggle MCP call ("requires approval, but approval policy is never"), and it runs only hooks you have already trusted. Trust the hooks once in the TUI (/hooks), and approve the muggle tools in ~/.codex/config.toml:

  [plugins."muggle@muggle-works".mcp_servers.muggle]
  default_tools_approval_mode = "approve"

  --dangerously-bypass-hook-trust skips the trust step for one run. It is meant for automation that already vets hook sources.

Registering only the MCP server, without the plugin? Keep the timeout, or local runs are cut off at a minute:

[mcp_servers.muggle]
command = "muggle"
args = ["serve"]
tool_timeout_sec = 1800
startup_timeout_sec = 120
env_vars = ["MUGGLE_MCP_PROMPT_SERVICE_TARGET"]

muggle doctor checks whichever of the two you set up.

Troubleshooting

unauthorized_client during login — the MCP server is pointed at one environment and you're authenticating against another. Fix MUGGLE_MCP_PROMPT_SERVICE_TARGET and restart the client.

Browser runner not found

muggle setup --force
muggle doctor

Authentication keeps expiring

muggle logout
rm ~/.muggle-ai/oauth-session.json ~/.muggle-ai/api-key.json
muggle login

The ecosystem

With both installed, muggle-ai-teams folds acceptance testing into each workflow step: test instructions written per slice at Plan, per-slice browser tests at Build, a full regression sweep at Verify, and results published and linked in the PR at Ship. Frontend slices get browser tests; backend-only slices are covered by unit tests, with the skip reasoned in writing.

Want it hosted, with nothing to configure? Muggle AI.

Contributing

pnpm install              # This repo is pnpm-only
pnpm run build            # tsup + plugin artifact
pnpm test                 # Tests impacted since origin/master, on 25% of cores (full suite in CI; MUGGLE_VITEST_FULL=1 forces it locally)
pnpm run lint             # Lint (auto-fix)
pnpm run typecheck        # Type check
pnpm run dev              # Watch mode

Verification gates, all run in CI:

pnpm run verify:plugin     # Plugin and marketplace metadata agree
pnpm run verify:contracts  # CLI/MCP/plugin/skill surface contracts hold
pnpm run verify:skill-deps # Skill dependencies stay one-way
pnpm run verify:signatures # Shipped artifacts are signed
pnpm run verify:upgrade-experience        # Existing-user upgrade still works
pnpm run verify:electron-release-checksums

Codex has its own layers, from static contracts up to model-driven evals. See docs/compatibility-strategy.md:

CODEX_BIN=$(command -v codex) pnpm run test:codex   # The plugin under the real Codex CLI, against a scripted model

Repository layout

muggle-ai-works/
├── plugin/               # Claude Code, Codex and Cursor plugin — source of truth
│   ├── skills/           #   Skill definitions (muggle-do, muggle-test, _shared/vcs, …)
│   ├── hooks/            #   Session hooks (codex-hooks.json is generated from hooks.json)
│   ├── .mcp.json         #   MCP server config
│   ├── .codex-mcp.json   #   The same server, with Codex's timeouts and env passthrough
│   └── .codex-plugin/    #   Codex manifest
├── src/
│   ├── cli/              # CLI entrypoint
│   └── server/           # MCP server — tool registration, stdio transport
├── packages/
│   ├── mcps/             # Tool registries, schemas, services
│   ├── commands/         # CLI command contracts
│   └── workflows/        # Workflow contracts
├── scripts/              # Build, verification, postinstall
├── config/compatibility/ # Surface contract baselines
├── internal/             # Maintainer-only skills (not published)
├── .claude-plugin/       # Marketplace catalog (Claude Code)
└── .agents/plugins/      # Marketplace catalog (Codex)

Releases

Two independent tag tracks: vX.Y.Z publishes @muggleai/works to npm; electron-app-vX.Y.Z publishes browser-runner binaries consumed by muggle setup and muggle upgrade.

Maintainers cut releases with the repo-local /mrelease skill rather than tagging by hand — CI can otherwise publish a version that disagrees with the checked-in manifests.

Agents pick tools by reading descriptions, so that text is tuned deliberately rather than written once. internal/skills/optimize-descriptions/SKILL.md documents the five layers of agent-facing text, where each lives, and how to build and run trigger eval sets against them. It is maintainer-only and ships in neither the npm package nor the plugin.

License

Apache 2.0. Use it, fork it, make it yours.

If it saves you a bad merge, a star helps others find it.