ResiliReplay

Turn a failed tool call from a supported coding agent into sanitized, deterministic, executable regression evidence.

MCP Inspector shows what a server does. ResiliReplay proves what happens when it fails, whether it recovers safely, and whether that recovery stays fixed. It is a local-first reliability layer, not another agent, observability dashboard, LLM evaluator, sandbox, or security certification.

npx --yes resilireplay@0.6.0 demo
npx --yes resilireplay@0.6.0 connect --agent auto --dry-run
npx --yes resilireplay@0.6.0 mcp serve

Static PNG · genuine transcript · self-contained regression · product site

The MCP Reliability Standard

The public working standard defines the minimum evidence for a bounded MCP reliability claim: pinned identity and authority, a clean control, deterministic fault, expected-failure negative control, bounded recovery, integrity hashes, executable regression when causal failure exists, and verified cleanup. It is an open testing convention—not the MCP specification, certification, security testing, or an endorsement.

Read the standard · run the five-minute MCP test · compare seven executed profiles · contribute a case

Static PNG · verified transcript · campaign · generated regression

Related MCP server: repo-seatbelt

Try it in under five minutes

Requirements: Node.js 22 or 24. Previewing is side-effect free; applying requires confirmation, backs up existing files, and leaves capture off.

npx --yes resilireplay@0.6.0 connect --agent auto --dry-run
npx --yes resilireplay@0.6.0 connect --agent claude-code
npx --yes resilireplay@0.6.0 connect --agent codex
npx --yes resilireplay@0.6.0 connect --agent hermes

npx --yes resilireplay@0.6.0 capture start
# reproduce one safe supported tool failure in the agent
npx --yes resilireplay@0.6.0 capture last
npx --yes resilireplay@0.6.0 capture stop
npx --yes resilireplay@0.6.0 capture generate-test

connect --dry-run prints the exact repository-local file operations and hashes without writing files or starting a process. Apply preserves unrelated settings, installs the portable skill, and records a recoverable backup. Use resilireplay connect --rollback to restore the latest backup. Capture is opt-in and passive: hooks never inject a fault or retry a failed operation. Hermes intentionally stages reviewable repository files and prints the official hermes mcp add follow-up instead of silently editing the global Hermes profile.

The in-agent request is:

Use ResiliReplay to capture this session’s tool failures, explain the recovery boundary, and turn the last supported failure into a regression test.

What is genuinely supported?

The v0.6 framework layer separates runtime proof from protocol fixtures and documentation:

pnpm demo:frameworks
pnpm exec resilireplay adapter detect --package @langchain/langgraph
pnpm exec resilireplay adapter doctor openai-agents

See the framework quick starts and support policy.

Coding-agent and MCP surfaces inherited from v0.5 are re-gated in v0.6:

Evidence labels are defined in the compatibility matrix. Vendor names describe factual interoperability and do not imply endorsement.

Claude Code plugin

The repository is a Claude Code marketplace, so the public install path is:

/plugin marketplace add aliengineering-byte/resilireplay
/plugin install resilireplay@resilireplay

The plugin contains .claude-plugin/plugin.json, stable PostToolUse, PostToolUseFailure, and Stop hooks, the portable skill, and the stdio MCP registration. The installed hook accepts stdin only, writes no stdout, validates CLAUDE_PLUGIN_ROOT, uses CLAUDE_PLUGIN_DATA only as vendor-owned writable state, and remains inert until capture is armed. See plugin operations and rollback.

Codex plugin

Codex discovers the repo marketplace at .agents/plugins/marketplace.json; the CLI test path is:

codex plugin marketplace add aliengineering-byte/resilireplay
codex plugin add resilireplay@resilireplay

The plugin contains .codex-plugin/plugin.json, the same skill and MCP server, and a bundled PostToolUse/Stop hook runtime. It distinguishes non-zero shell results, MCP errors/success, file edits, interruptions, duplicates, oversized payloads, secret-shaped input, and unsupported hosted-tool fixtures without assuming every delivered event succeeded. PLUGIN_ROOT is immutable; PLUGIN_DATA is vendor-owned writable state. See plugin trust boundaries.

Universal ResiliReplay MCP server

npx --yes resilireplay@0.6.0 mcp serve

The default transport is stdio. Nine annotated tools cover status/version, fault discovery, sanitized target inspection, campaign validation, passive capture start/stop, last-failure evidence, regression generation, and explicitly confirmed campaign execution. Read-only, destructive, idempotent, and open-world annotations are declared. Regression writes require the exact evidence SHA-256; campaign execution requires the exact reviewed campaign SHA-256 and retains the existing allowlists and remote-target boundary.

Stable schemas and adapter contract

The canonical integration engine is @resilireplay/agent; vendor hooks are thin adapters. Public v1 schemas are:

• resilireplay.agent-event/v1

• resilireplay.capture-session/v1

• resilireplay.failure-evidence/v1

• resilireplay.adapter-manifest/v1

• resilireplay.framework-event/v1

• resilireplay.adapter-template/v1

Create and verify an adapter without changing the engine:

npx --yes resilireplay@0.6.0 adapter init my-agent-adapter
npx --yes resilireplay@0.6.0 adapter verify ./my-agent-adapter

“ResiliReplay Compatible” means only that the published conformance suite passed. Read the adapter contract, minimal adapter, and badge rules.

Existing v0.4 workflows remain supported

resilireplay adopt --config ./mcp.json --dry-run
resilireplay studio --open
resilireplay campaign validate campaign.yml
resilireplay campaign run campaign.yml --confirm-tools <reviewed-sha256>
resilireplay campaign approve runs/candidate --output baselines/main.json
resilireplay campaign compare runs/current --baseline baselines/main.json
resilireplay record --output runs/agent/trace.jsonl -- node agent.js
resilireplay inject --trace runs/agent/trace.jsonl --scenario malformed-json --seed 42 --output runs/agent/failed.jsonl
resilireplay replay --trace runs/agent/failed.jsonl --report-dir runs/agent/report
resilireplay generate-test --trace runs/agent/failed.jsonl --output runs/agent/regression
resilireplay mcp audit --inspector-config ./mcp.json --server my-server --dry-run

Campaign exit codes remain 0 pass, 1 reliability failure/regression, 2 usage, 20 invalid schema, 21 target/authorization, 22 execution, 23 cancelled/incomplete, and 24 integrity failure. See the v0.5 migration guide, campaign schema, and MCP Inspector integration.

Privacy, security, and measured bounds

• Capture is off by default. No telemetry, cloud account, billing, background upload, raw prompt, full transcript, environment value, authorization header, token, unrestricted tool body, or personal path is persisted by default.

• Session and tool-call identifiers are one-way SHA-256 projections. Bodies become hashes; summaries are redacted and capped at 512 characters.

• Capture is capped at 20,000 events and 32 KiB per normalized event. State and evidence use atomic replacement; the locked append-only journal repairs an interrupted trailing record.

• Hook writers are serialized, duplicate tool-call outcomes are ignored, symlink/junction escapes fail closed, and generated regressions refuse overwrite.

• Hooks never execute a target, retry a call, or inject a failure. Explicit campaigns retain their review, allowlist, hash confirmation, retry, and cleanup controls.

On the recorded Windows Node 24 release workload, 20,000 synthetic normalized events took 898 ms, produced 13,414,112 bytes, and used a measured 100,171,776-byte RSS delta including the 20,000-event input array. In-process single-event capture measured 14.94 ms median and 18.43 ms p95 across 100 samples; startup was 20.48 ms and cleanup 31.55 ms. These workloads are not equivalent to end-to-end agent latency and are not comparative claims. CI regenerates the report with pnpm agent:gates.

Read SECURITY.md, THREAT_MODEL.md, limitations, and release evidence.

Architecture

Development

corepack enable
pnpm install --frozen-lockfile
pnpm quality
pnpm demo:frameworks
pnpm test:e2e
pnpm site:test
pnpm release:gates
pnpm agent:gates

ResiliReplay is Apache-2.0 licensed. Contributions must remain deterministic, bounded, secure by default, and covered by tests. See CONTRIBUTING.md, CODE_OF_CONDUCT.md, public adoption policy, and the ecosystem page.

Built and maintained by Ali.