Chaos-MCP

On-demand micro-mutation sandbox for AI test verification — maps holes in unit tests by running isolated mutation testing via the Model Context Protocol.

Pre-release / in active development. Chaos-MCP is not yet published to npm. The source is public on GitHub — install from source (see Installation). Any npm install -g / npx commands in this README describe the planned published experience and do not work yet.

Chaos-MCP is an MCP (Model Context Protocol) server that exposes three tools — audit_code_resilience (audit a single file), triage_test_coverage (rank a whole tree weakest-first), and estimate_audit (cheap pre-flight mutant count / timing estimate) — which run isolated mutation testing against your source to find weaknesses in the local test suite. It intentionally injects logical faults (like changing > to >=) and checks whether your tests catch them. Surviving mutants indicate test coverage holes.

Features

• 4 Languages Supported — TypeScript/JavaScript (StrykerJS), Python (cosmic-ray), Rust (cargo-mutants), PHP (Infection)

• Sandbox Isolation — all mutation runs execute in temporary directories; your real workspace is never touched

• Pinned Container Runners — release-matched OCI images provide all four mutation engines without installing them on the host

• Auto-Detection — automatically detects project type, test runner, and workspace root

• Async Subprocesses — all mutation-tool execution uses async execFile/exec (subprocess runs never block the event loop; the one-time sandbox copy is synchronous)

• Rich Tool Schema — supports line scoping, mutator denylists, concurrency control, dry-run mode, incremental runs, and output format selection

• Pre-flight Estimation — estimate_audit gives a fast mutant count (exact for Rust, approximate for others) and optional timing estimate before you commit to a full run

• Gate Mode — pass minScore to audit_code_resilience or triage_test_coverage to get a machine-readable pass/fail field for CI pipelines

• Cross-Platform — works on macOS, Linux, and Windows (with junction fallback for symlinks)

Related MCP server: Mutmut MCP

Installation

While in development, the only supported install path is from source — clone the repo, build, and register the built entrypoint with your MCP client.

git clone https://github.com/AraneaDev/Chaos-MCP.git
cd Chaos-MCP
npm install
npm run build      # compiles to build/index.js

Register it with an MCP client (Claude Code example):

claude mcp add chaos-mcp -- node /absolute/path/to/ChaosMCP/build/index.js

Planned (not available yet): once published, install will be npm install -g chaos-mcp or run on demand via npx chaos-mcp. These do not work until the package ships to npm.

Prerequisites — language mutation tools

Native mode (the default) shells out to mutation engines installed on the host. Install only the engine(s) for the languages you audit. Alternatively, enable container execution to use the release-matched, pinned engines without installing them on the host. Missing native tools return a clear error naming the exact install command.

Notes:

• In native mode, the tool must be on PATH (or, for StrykerJS, resolvable from the target project's node_modules), and its language toolchain must be installed.

• PHP / Infection: set failOnWarning="true" in your phpunit.xml. Infection writes a PHPUnit config per mutant with stopOnDefect="true", so the suite stops as soon as a mutant looks killed — but a PHP warning is a defect without being a failure, so under failOnWarning="false" a mutant that makes an earlier test warn stops the run with exit 0 and is reported as survived before the asserting test runs. Scores are only ever depressed by this, never inflated. Chaos-MCP reads your PHPUnit config and attaches a fidelityNote to any PHP result that reports survivors while the setting is off.

• Python / cosmic-ray (native mode): on modern distros a bare pip install cosmic-ray is blocked by PEP 668 ("externally-managed-environment"); use pipx install cosmic-ray or an activated virtualenv. Chaos-MCP generates cosmic-ray's config and runs baseline → init → exec → dump in the sandbox. Use testSelection and excludeOperators to keep large audits tractable.

• These engines run inside the sandbox against a copy of your workspace; Chaos-MCP never installs or modifies anything in your real project.

For container mode, install Docker or Podman. Both runtimes normally pull a missing image while creating the first audit container, but pre-pulling avoids making a large download compete with the container startup timeout:

CHAOS_MCP_TAG="v$(node -p "require('./package.json').version")"
docker pull "ghcr.io/araneadev/chaos-mcp-typescript:${CHAOS_MCP_TAG}"
docker pull "ghcr.io/araneadev/chaos-mcp-python:${CHAOS_MCP_TAG}"
docker pull "ghcr.io/araneadev/chaos-mcp-rust:${CHAOS_MCP_TAG}"
docker pull "ghcr.io/araneadev/chaos-mcp-php:${CHAOS_MCP_TAG}"

# From a source checkout:
node build/index.js --container-doctor

Use podman pull instead when "runtime": "podman" is configured. Each Chaos-MCP release selects its matching vX.Y.Z image tags automatically.

Quick Start

1. Start the Server

Normally your MCP client launches the server for you (see Installation). To run it directly from a source checkout:

# From the repo root, after `npm run build`
npm start                                  # → node build/index.js
node build/index.js --verbose              # diagnostic logging to stderr
node build/index.js --config ./chaos-mcp.config.json

2. Call the Tool from Your MCP Client

The primary tool is audit_code_resilience (the batch tool triage_test_coverage is documented below; the lightweight pre-flight tool estimate_audit is documented below).

Minimal example:

{
  "filePath": "src/utils/math.ts"
}

Full example with all options:

{
  "filePath": "src/utils/math.ts",
  "timeoutMs": 120000,
  "lineScope": { "start": 10, "end": 80 },
  "mutatorDenylist": ["StringLiteral"],
  "concurrency": 4,
  "incremental": true,
  "ignorePatterns": ["fixtures/", "snapshots/"],
  "outputFormat": "text",
  "enrich": false,
  "maxSurvivors": 20,
  "severityFloor": "medium"
}

Get enriched, severity-ranked guidance on survivors (on by default):

Enrichment is enabled by default. Each surviving / no-coverage line is augmented with four fields: a severity rating (high, medium, or low) based on the mutator's semantics (e.g. boundary operators and logical operators rank high), a why explanation of why the gap is dangerous, a hint describing the kind of test that would kill it, and a context snippet of the surrounding source lines. Survivors are re-ranked severity-first so the most critical gaps appear first. To disable enrichment and return the plain unranked output, pass "enrich": false.

TypeScript targets produce the richest output because StrykerJS exposes per-mutant operator detail; Python (cosmic-ray) targets also produce severity-ranked output, mapping the tool's authoritative operator name to a canonical category; targets whose tool can't expose a per-mutant operator fall back to severity: "unknown" with a generic why/hint.

Cap and filter the survivor list:

{
  "filePath": "src/utils/math.ts",
  "maxSurvivors": 5,
  "severityFloor": "high"
}

maxSurvivors caps how many survivor (and no-coverage) line groups are returned after severity ranking (default: 10; configurable via defaultMaxSurvivors). Hidden groups are counted in survivorsTruncated / noCoverageTruncated in the output. severityFloor drops groups below the given severity level (requires enrichment, which is on by default); dropped groups are counted in survivorsFiltered / noCoverageFiltered.

Scope to just your uncommitted changes:

{
  "filePath": "src/utils/math.ts",
  "diffBase": "HEAD"
}

Mutation-tests only the lines you've changed since the last commit.

Verify your new tests killed the previous survivors:

{
  "filePath": "src/utils/math.ts",
  "baseline": { "survivors": [{ "line": 42, "mutators": { "ConditionalExpression": 1 } }] }
}

Re-runs only the baseline lines and reports which previously-uncaught mutants are now killed:

{
  "mode": "verify",
  "baselineTotal": 1,
  "killedCount": 1,
  "nowKilled": [{ "line": 42, "mutator": "ConditionalExpression" }],
  "stillSurviving": [],
  "newSurvivors": []
}

3. Interpret the Results

The output is bundled and deduplicated to stay token-efficient: mutants are grouped by line (with a per-line count of each mutator type), survivors (tests ran but didn't catch) and noCoverage (no test reached the mutant) are reported separately at line+mutator granularity, and the explanatory note appears once instead of being repeated for every mutant. Because the split is per-mutator, the same line can appear in both lists (e.g. a live expression that survived next to an unreachable fallback that no test reached). Survivors and no-coverage entries also include a changes sample — a capped, deduped list of original → mutated edits — for TypeScript and Rust targets (best-effort; absent for Python, which doesn't expose per-mutant detail). When diffBase is used, the output may include a scopeNote (a top-level JSON field / a Scope: text line) reporting scoping decisions — e.g. a skipped run when nothing changed, or a whole-file fallback for Python/Rust targets.

JSON output (default — emitted as a single compact line):

{
  "target": "src/utils/math.ts",
  "mutationScore": "91.67%",
  "summary": { "total": 12, "killed": 11, "survived": 1, "worstSeverity": "high" },
  "survivors": [
    {
      "line": 42,
      "mutators": { "ConditionalExpression": 1 },
      "changes": ["a > b → a >= b"],
      "severity": "high",
      "why": "a branch condition was forced to a constant; a test passed without exercising both arms.",
      "hint": "add tests that take BOTH the true and the false branch.",
      "context": ["41: if (a > b) {", "42:   return a;", "43: }"]
    }
  ],
  "noCoverage": [],
  "suggestedTestFile": { "path": "src/utils/__tests__/math.test.ts", "exists": false },
  "note": "survivors: mutants your tests ran but did not kill. noCoverage: mutants no test reached (per line+mutator, so a line may appear here and in survivors). mutators = type→count. Add or strengthen tests targeting these. changes = sampled original→mutated edits for that line (capped)."
}

The tool response also carries a structuredContent field (in addition to the standard text content block) so MCP clients that support it can consume the data directly without parsing JSON from text. The text block is retained for compatibility with clients that read content[0].text.

suggestedTestFile is included when there are survivors or no-coverage entries (i.e. when the mutation score is below 100%), pointing to the conventional test file path for the audited source file (e.g. src/utils/__tests__/math.test.ts for src/utils/math.ts). The exists flag indicates whether the file already exists on disk.

Text output ("outputFormat": "text"):

Chaos-MCP Audit Report: src/utils/math.ts
Mutation score: 91.67% (11/12 killed, 1 survived)
Survivors (line: mutators):
  42: ConditionalExpression  (a > b → a >= b)
Add or strengthen tests targeting these lines to kill the survivors.

Tool Parameters

See CONTRIBUTING.md for development setup and the full parameter semantics.

State & the verify loop

Verify loop via runId

Every successful, non-verify audit_code_resilience call returns a runId (an 8-character id) in its JSON output. Use it to re-verify without copying the full baseline object:

1. Audit: { "filePath": "src/utils/math.ts" } → response includes "runId": "a1b2c3d4".

2. Fix or add tests.

3. Verify: { "filePath": "src/utils/math.ts", "runId": "a1b2c3d4" } → reports which previously-surviving mutants are now killed.

runId is mutually exclusive with baseline, diffBase, and lineScope. The baseline cache lives in os.tmpdir()/chaos-mcp-runs/ and is ephemeral (default TTL: 24 h; default max: 200 entries). Passing an unknown or expired runId returns an error.

triage_test_coverage also mints and returns a runId per ranking row, so you can drill into a weak file and immediately verify after fixing its tests.

Suppressing equivalent mutants

Some mutants are equivalent — logically identical to the original under all possible inputs — and cannot be killed by any test. Suppress them so they stop appearing in the output and stop dragging down the score:

{
  "filePath": "src/utils/math.ts",
  "suppress": [
    { "line": 99, "mutator": "StringLiteral", "reason": "guard always true for this type" }
  ]
}

Suppressed mutants are:

• Persisted to <workspaceRoot>/.chaos-mcp/suppressions.json (keyed by workspace-relative file path).

• Auto-excluded from every future audit and triage call for that file — no flag needed.

• Removed from the score denominator — mutationScore rises and the output field suppressedCount tells you how many were excluded.

• Excluded from verify mode — suppressed mutants won't appear as "still surviving".

To undo a wrong suppression:

{
  "filePath": "src/utils/math.ts",
  "unsuppress": [{ "line": 99, "mutator": "StringLiteral" }]
}

.gitignore or commit? Add .chaos-mcp/ to .gitignore if the suppression list is personal, or commit it to share the equivalent-mutant list with the team. Suppression keys are workspace-relative, so the file is portable across machines.

Staleness caveat: entries are keyed by file + line + mutator. Edits that shift line numbers can stale an entry. Each entry records an optional reason and an addedAt timestamp so you can audit and prune the list over time.

Config keys for state

Batch Triage — triage_test_coverage

A second tool ranks where your test suite is weakest across many files in one call.

{ "paths": ["src/utils", "src/index.ts"], "maxFiles": 25 }

Directories are recursively expanded to supported source files (test files skipped), audited in bounded parallel (default max(1, min(4, cpus-1)) files at a time; capped at maxFiles; precedence maxFiles arg → defaultMaxFiles config → 25), and ranked weakest-first by mutation score:

{
  "mode": "triage",
  "summary": { "filesDiscovered": 30, "filesAudited": 25, "filesSkipped": 5, "filesErrored": 0 },
  "ranking": [
    {
      "file": "src/a.ts",
      "mutationScore": "62.50%",
      "total": 16,
      "killed": 10,
      "survived": 5,
      "noCoverage": 1
    }
  ],
  "errors": [],
  "note": "Ranked weakest-first by mutation score. Drill into a file with audit_code_resilience for survivor detail."
}

The tool response carries a structuredContent field (in addition to the text block) so MCP clients can consume the ranked payload directly without parsing JSON. The outputSchema on the tool definition describes the payload shape.

Drill into a weak file with audit_code_resilience for per-mutant survivor detail.

PR-diff scan — diffBase:

Pass diffBase to limit the triage to files changed in a PR or branch. paths becomes optional in this mode:

{ "diffBase": "main" }

diffBase alone audits every changed supported source file in the workspace (relative to main via merge-base). Passing both limits the scan to changed files under those paths:

{ "diffBase": "main", "paths": ["src/utils"] }

TypeScript files are mutated only on the changed lines; Python and Rust files run whole-file (a per-file scopeNote is included in the ranking row).

Inline survivor detail — survivorsPerFile:

{ "paths": ["src"], "survivorsPerFile": 3 }

survivorsPerFile (default 0, scores-only) inlines the top-N severity-ranked, enriched survivor groups into each ranking row so you can triage and inspect in one call. Set it to 0 for the compact leaderboard; raise it when you want to see the worst gaps immediately.

Parallel file auditing — fileConcurrency:

{ "paths": ["src"], "fileConcurrency": 8 }

fileConcurrency controls how many files are audited in parallel (default max(1, min(4, cpus-1)); range 1–64). When fileConcurrency > 1 and the file is TypeScript, each StrykerJS run's worker count is automatically capped (floor((cpus-1) / fileConcurrency)) so total CPU use stays near the core count rather than oversubscribing. Other languages run their mutation tool without a worker-count override (they ignore the concurrency cap).

Parameters:

Pre-flight Estimate — estimate_audit

Before committing to a full mutation run, use estimate_audit to check how many mutants a file will produce and (optionally) how long the run will take. It never runs the mutation test cycle by default.

{ "filePath": "src/utils/math.ts" }

Output:

{
  "target": "src/utils/math.ts",
  "language": "typescript",
  "mutants": 47,
  "fidelity": "approx",
  "basis": "source heuristic: 23 constructs",
  "note": "Approximate mutant count from a source-parse heuristic; the real audit may differ. Run audit_code_resilience for exact results."
}

With timing (withTiming: true): runs the test suite once to measure a baseline, then estimates total wall-clock time as mutants × baseline / concurrency. This provisions a sandbox and counts against your machine's resources — use it when you want a time budget before a large audit.

{ "filePath": "src/utils/math.ts", "withTiming": true }

Additional output fields when withTiming: true:

{
  "baselineMs": 4200,
  "estimatedMs": 197400,
  "concurrency": 1
}

Fidelity

For Rust, the estimate is exact because cargo mutants --list enumerates every planned mutant without running tests. For all other languages the count is approximate — a lightweight heuristic over the source AST; the actual audit may differ. Run audit_code_resilience for exact results.

If cargo-mutants is not installed, the Rust path falls back to the heuristic and reports fidelity: "approx" with a note.

Parameters

Use case

Call estimate_audit first when you are unsure whether a file is too large to audit interactively:

1. estimate_audit { "filePath": "src/big.ts" } → 300 mutants, approx.

2. Consider scoping with lineScope or diffBase, or scheduling the full run with a longer timeoutMs.

3. audit_code_resilience { "filePath": "src/big.ts", "diffBase": "HEAD" } → audits only your changed lines.

Gate Mode — minScore

Both audit_code_resilience and triage_test_coverage accept a minScore parameter (0–100). When the mutation score falls below the threshold, the result reports the gate as failed. A failing gate is never an error — it is a data field for an agent or CI pipeline to read and act on.

Gate on a single file

{ "filePath": "src/utils/math.ts", "minScore": 80 }

If the mutation score is below 80, the output includes:

{ "gate": { "minScore": 80, "passed": false } }

If the score meets or exceeds the threshold, gate.passed is true. The field is absent when minScore is not provided.

The gate uses the suppression-adjusted mutation score (i.e. equivalent mutants excluded via suppress are not counted against the denominator).

Gate on a triage run

{ "paths": ["src"], "minScore": 75 }

Each ranking row gains a passed field. The top-level output includes:

{
  "gate": {
    "minScore": 75,
    "passed": false,
    "failingFiles": ["src/utils/math.ts", "src/parser.ts"]
  }
}

gate.passed is false if any file's score is below minScore. failingFiles lists the workspace-relative paths that did not pass. Files that errored during triage are reported in errors[] and do not affect the gate.

CI use case

# Fail CI if any audited file scores below 80%
mcp call triage_test_coverage '{"paths":["src"],"minScore":80}' \
  | jq -e '.gate.passed'

An agent or CI script reads gate.passed and decides whether to block the build, open an issue, or continue. The tool call itself always succeeds (never isError) regardless of the gate outcome.

Configuration

Create a chaos-mcp.config.json in your workspace root for default settings:

{
  "defaultTimeoutMs": 300000,
  "mutatorDenylist": ["StringLiteral"],
  "concurrency": 4,
  "defaultMaxFiles": 25,
  "defaultMaxSurvivors": 10,
  "defaultSeverityFloor": "medium",
  "defaultFileConcurrency": 4,
  "container": {
    "mode": "auto",
    "runtime": "docker",
    "cpus": 2,
    "memoryMb": 4096
  }
}

Tool call arguments override config defaults.

Container execution

Container mode removes the need to install StrykerJS, Cosmic Ray, cargo-mutants, or Infection on the host. Chaos-MCP starts one hardened, short-lived container per audit, mounts the temporary sandbox at /workspace, and runs both prebuild and mutation commands in that session. The real workspace is never mounted. Recognized dependency trees linked into the sandbox may be mounted separately and read-only, as described below.

{
  "container": {
    "mode": "container",
    "runtime": "docker",
    "network": "bridge",
    "cpus": 2,
    "memoryMb": 4096,
    "pidsLimit": 512,
    "startupTimeoutMs": 60000
  }
}

Modes:

• native (default) preserves the existing host-subprocess behavior.

• container requires Docker or Podman and fails clearly when unavailable.

• auto uses containers when the configured runtime is reachable and otherwise falls back to native. Image or project failures do not silently fall back.

Container settings:

The images pin the language runtime and mutation engine, while the project still supplies its own test dependencies. Existing sandbox dependency directories (node_modules, .venv/venv, and vendor) are mounted read-only when they are symlinked from the host. Dependencies containing native extensions must be compatible with the selected Linux image; use an image override when the project requires another runtime or platform build. Chaos-MCP selects release-matched GHCR images for all four languages by default; the optional images map accepts per-language tags or digest-pinned references for private mirrors and custom runtimes.

The official images are published for Linux AMD64 and ARM64:

The server never installs target-project dependencies. It reuses recognized dependency directories from the sandbox when available (node_modules, .venv/venv, and vendor), mounting host-linked directories read-only. Install the target project's dependencies before auditing it.

Containers run with a read-only root filesystem, all Linux capabilities dropped, no-new-privileges, a private temporary filesystem, and configurable CPU, memory, and PID limits. Resource usage defaults to a conservative two CPUs and 4096 MiB of memory. The entire container is forcibly removed on timeout, cancellation, normal completion, or engine failure.

Network mode is part of the audit's isolation boundary. The bridge default allows outbound access for tests or dependency resolution; use "network": "none" for untrusted code or offline audits that do not require network access. Avoid host networking unless the target project explicitly requires it and you accept the reduced isolation.

Check runtime connectivity and whether all four configured images are already present without pulling anything:

node build/index.js --container-doctor

The doctor exits non-zero when the runtime is unavailable or any configured image is missing. Pull the reported image, or set a matching entry in container.images, and run it again. startupTimeoutMs only governs runtime probing and container startup; defaultTimeoutMs and per-tool timeoutMs govern the mutation audit itself.

Enabling prebuildCommand

The prebuildCommand tool argument runs an arbitrary shell command inside the sandbox, which can reach outside it. It is disabled by default. Enable it explicitly with "allowPrebuild": true in chaos-mcp.config.json, or by setting the CHAOS_MCP_ALLOW_PREBUILD=1 environment variable. The auto-detected prebuild for Rust (cargo check) runs without this flag.

Auditing workspaces outside the working directory

By default Chaos-MCP only audits files beneath the directory the process was launched in: the workspace-root walk stops at process.cwd(), and the sandbox refuses to copy anything that escapes it. For an MCP server — which is launched once with a fixed cwd — that means a server started in project A cannot audit project B at all.

Set CHAOS_ALLOWED_ROOTS to name additional roots, separated by the platform path delimiter (: on POSIX, ; on Windows):

{
  "mcpServers": {
    "chaos-mcp": {
      "command": "node",
      "args": ["/path/to/Chaos-MCP/build/index.js"],
      "env": { "CHAOS_ALLOWED_ROOTS": "/srv/project-b:/srv/project-c" }
    }
  }
}

A workspace is accepted when it is inside the working directory or inside one of these roots; everything else is still refused. Descendants of a listed root are included, siblings and parents are not. When roots nest (a monorepo and one of its packages), the innermost match bounds the root walk. Leaving the variable unset keeps the cwd-only behaviour exactly as before.

Supported Test Runners (Auto-Detected)

CLI Flags

chaos-mcp [flags]

  --version   Print version and exit
  --help      Show help text and exit
  --config    Path to a JSON config file
  --container-doctor
              Check runtime connectivity and whether all four configured
              container images are present without pulling them
  --verbose   Enable diagnostic logging to stderr

Protocol features

Progress notifications

When an MCP client includes a progressToken in a tool call's _meta field, Chaos-MCP emits notifications/progress events during the run. Clients that omit progressToken receive no notifications — there is zero overhead for clients that do not opt in.

Triage emits one notification per file as it completes:

Audit emits four coarse milestones:

Estimate does not emit progress notifications.

Cancellation

Cancelling an in-flight MCP request aborts the run cleanly:

• The abort signal propagates through the tool handler into RunOptions.signal and from there into the mutation engine subprocess, terminating it.

• The sandbox is always cleaned up even if cancellation occurs mid-run.

• The cancelled call returns "Operation cancelled." as a tool error rather than throwing.

All three tools (audit_code_resilience, triage_test_coverage, estimate_audit) respect cancellation.

Resources

The server exposes three static resources, discoverable via resources/list and readable via resources/read:

Prompts

The server exposes two prompts, discoverable via prompts/list and retrieved via prompts/get:

Development

npm run check         # Full CI pipeline: build + lint + format + test
npm run test:watch    # Watch mode for iterative development
npm run test:coverage # Tests with coverage report

The suite runs on every push/PR to main via CI (Node 22/24). v8 line/statement coverage of src/ sits at ~99%, and the source is additionally hardened by running Chaos-MCP against its own code — so the suite is graded by mutation score, not just line coverage.

See CONTRIBUTING.md for detailed development setup and contribution guidelines.

License

MIT — See LICENSE for details.

Links

• MCP Documentation

• StrykerJS

• cosmic-ray

• cargo-mutants

• Infection

• Changelog