Skip to main content
Glama

Recovery Probe

Stop assuming your code works. Test what actually happens.

Recovery Probe helps developers and AI coding agents discover bugs that ordinary tests can miss. It deliberately triggers failures in web and Electron applications, observes how the UI responds, and produces evidence you can use to reproduce, fix, and verify problems.

A failed request should not silently turn a populated screen into “No items found.” Recovery Probe helps you reproduce that failure and check whether the application recovers as expected.

CI npm License: MIT

Recovery Probe catches a broken retry flow and passes the fixed one

Try it

You need Node.js 22 or newer.

npm install --save-dev recovery-probe playwright
npx playwright install chromium
npx recovery-probe demo

Version 1.1.0 adds an experimental verification planner, row progress tracking, and expanded runtime observation. Install the stable release with the command above.

The next preview, 1.2.0-preview.1, adds an experimental scenario self-check: it validates registered scenarios statically and, on request, runs them a few times on a build you name, before you rely on them to verify a change. After publication, install it with npm install --save-dev recovery-probe@next playwright. Until then, use the source installation instructions.

Plan checks for a code change

Register existing scenarios in recovery-probe.registry.json, then run:

npx recovery-probe plan --changed-since main --write .recovery-probe/verify.json

The experimental planner selects registered checks, explains why, and lists missing coverage or expectations. It does not invent tests or declare a change correct. Review the plan, record a baseline against the before-change application, then use verify after the change. See planning and registry setup. Coding agents can use the matching plan_verification MCP tool.

The demo runs two versions of a small app. Both load fine. Only the fixed one survives a 503, a dropped connection and a broken JSON response.

Related MCP server: MCP Failure Lab

Check your own app

Web app. Start it, then run:

npx recovery-probe init http://localhost:3000

A browser opens. Sign in, go to the screen you care about, and click something that only shows up once its data has loaded. Recovery Probe finds the request behind it, fails it three ways, and tells you whether the data came back.

Electron app. Start it with --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222, then run npx recovery-probe init.

Electron app that loads data over IPC. Load the small development-only hook in the main process. npx recovery-probe doctor prints the code and the path to load it from. Restart the app with the inspector on, then run npx recovery-probe init: it lists the hook's channels, asks which one loads the screen, and fails it in a few ways.

Each run saves an HTML and JSON report. Run the same command again to repeat the check, or add --fresh to set it up again. If something's missing, npx recovery-probe doctor says what and how to fix it.

More

To

Run

Docs

Fail the read behind a screen without writing assertions

discover

Discovery

Catch stale screens and rows stuck in an old state, without faults

observe

Observe

See what users get when Save or Send fails

action

Action check

Fail an SDK's or a main process's own HTTPS connections

net

Network faults

Find code that turns a failed read into "no data"

suspects

Suspects

Choose the checks a change needs (experimental)

plan

Plan

Check registered scenarios before relying on them (experimental)

self-check

Self-check

Prove a change fixed it

compare, watch, verify

Compare, Watch, Verify

Use it from a coding agent

recovery-probe mcp is an MCP server. Any agent that supports MCP can use it to fail a read and look at the screen before it claims the code handles errors. Most agents take this config:

{
  "mcpServers": {
    "recovery-probe": { "command": "npx", "args": ["recovery-probe", "mcp"] }
  }
}

Other config formats and the list of tools are in MCP.

Use it in your tests

withFault fails matching requests inside a Playwright test:

import { test, expect } from '@playwright/test'
import { withFault } from 'recovery-probe'

test('profile recovers after a 503', async ({ page }) => {
  const stats = await withFault(
    page,
    { match: '**/api/profile', kind: 'http-error', status: 503 },
    async () => {
      await page.goto('http://127.0.0.1:3000/profile')
      await page.getByRole('button', { name: 'Retry' }).click()
      await expect(page.getByTestId('profile-name')).toHaveText('Test User')
    },
  )
  expect(stats.applied).toBe(1)
})

If nothing matches, withFault throws FAULT_NOT_TRIGGERED instead of passing. For code that takes a fetch function there's createFaultFetch. See API.

How it decides

  • Every faulted run has a healthy run right before it, on the same screen, and the report compares the two.

  • A result is a pass, a failure or inconclusive. A fault that never fired, an unstable baseline or a path it can't reach is inconclusive, never a pass.

  • Some findings, like SILENT_EMPTY, are leads. They tell you where to look, not that the code is wrong.

  • Exit codes: 0 passed, 1 a check failed, 2 needs review or inconclusive, 3 wrong usage or config, 4 couldn't run.

Where it works

  • Web apps in Chromium, either launched by Recovery Probe or attached through a local debugging port.

  • Electron apps, over HTTP in the renderer or over IPC with the hook.

  • Native HTTPS clients that use HTTPS_PROXY, such as an SDK in the main process. Only whole connections can fail there.

  • Android WebViews, experimental.

It doesn't simulate a whole device going offline, WebSocket drops, token expiry, sleep or slow networks.

Privacy

Reports stay on your machine and there's no telemetry. Request and response bodies aren't recorded, and discovery and observation keep screen text only as hashes. Reports can still contain endpoint paths, selectors, text you picked as a marker and, with --screenshots, what was on screen, so check them before you share them. Nothing calls an AI service except the opt-in repair workflow.

Failed recovery checks can also save screenText: up to 200 characters of newly visible screen text, included in terminal output, reports, fix briefs and MCP results. These quotes are not redacted and can contain personal or sensitive data even without screenshots. Review these outputs before sharing them or passing them to an agent.

Only test apps and accounts you're allowed to, and keep debugging ports on localhost.

Compatibility in 1.x

The compatibility commitment covers documented, non-experimental package exports, CLI commands and options, exit codes, configuration fields, MCP tools and JSON report fields. Removing or renaming these interfaces, changing their types incompatibly, or changing their documented meaning requires a major version. Backward-compatible additions can ship in minor releases; fixes that restore documented behavior can ship in patch releases.

Consumers should tolerate additional JSON fields. Exact terminal wording, HTML layout, timing measurements and heuristic findings are not fixed contracts. Internal files under dist/ are not public imports; use the package's declared exports. Features explicitly labeled experimental, including Android WebView support and repair, may change in minor releases, with changes recorded in the changelog.

Upgrading to 1.0

1.0 removes the old command spellings desktop, web, ipc, --demo and --config. Each now stops with a usage error that names the commands that replaced it, such as windows, reads, check, discover, action and demo. Usage errors exit with 3 and setup failures with 4, where the old spellings exited with 2, so update CI scripts that read exit codes. demo and the JSON runner's check write report.json into the --out folder. Restart the development app after upgrading so its IPC hook matches the CLI. Public package imports and the hook's path are unchanged. See the changelog.

Docs

Guided start · Desktop and browsers · IPC · Discovery · Observe · Refresh · Network faults · Android · Suspects · Compare · Watch · Plan · Self-check · Verify · MCP · API · JSON runner · CI · Repair · Changelog

Contributing

See CONTRIBUTING.md. Report security issues as described in SECURITY.md.

License

MIT © 2026 Bojan Kovachki

Available Tools

12 tools
check_read_recoveryCheck that a page recovers when a read failsA

Loads the page healthy, then fails one GET endpoint with each fault kind and checks that the ready marker comes back: by itself (recovery "automatic") or after clicking retrySelector ("retry"). Returns an outcome per fault, why the healthy baseline failed if it did, the text the screen showed under a fault it did not recover from, and the report folder, which includes fix-brief.md and reproduce.mjs.

Inputs: url (required), endpoint (required), readySelector (required), times, +6 more.

compare_reportsCompare reports from before and after a changeA
Read-only

Compares reports of the same check from before and after a change, such as main and a branch, and returns a verdict (IMPROVED, PARTLY_IMPROVED, REGRESSED, MIXED, UNCHANGED, INCONCLUSIVE or NOT_COMPARABLE), what happened to each problem, and a Markdown block for a pull request. A problem counts as gone only when a complete run after the change checked it, so a run that stopped early is never a fix. Give several reports per side for repeated runs.

Inputs: after (required), before (required), afterLabel, beforeLabel.

find_page_readsFind the reads behind a pageA
Read-only

Opens a page in headless Chromium and lists the JSON GET requests it makes while it loads. Use it to choose the endpoint for check_read_recovery. Only successful page fetch and XHR reads are seen; native, IPC and service-worker traffic is not.

Inputs: url (required), durationMs, storageState.

find_suspectsFind failure handling that can lose dataA
Read-only

Reads JavaScript and TypeScript sources, without running them, for two patterns: UNKNOWN_AS_EMPTY (a failed or unexpected read becomes an empty collection, so "unknown" looks like "none") and FAILURE_ONLY_LOGGED (a read stores its data when it succeeds, but its failure is only logged). Each suspect names the read to fail and the check that proves or dismisses it. Use changedSince "main" to scan only what a branch changed. Needs TypeScript in the project.

Inputs: paths, changedSince, includeTests.

network_fault_summaryShow what the network faults didA
Read-only

Reports how many connections went through the proxy and, for the armed rules, how many each matched and faulted. A rule that faulted nothing means the app did not use the proxy or the host name is wrong.

Inputs: no named parameters.

plan_verificationPropose the checks that verify a code changeA

Experimental. Before recording a baseline, proposes which existing Recovery Probe checks verify a change: it matches the files changed since a Git reference (committed, staged, unstaged, untracked, deleted, renamed) against the project scenario registry (recovery-probe.registry.json) and the suspects on the changed lines. Each check says which file selected it, what it checks, where its expectation came from and what it does not cover. It also lists suggestions to review, changed files with no known check, and what needs a person: open questions, missing expectations, invalid entries. It runs nothing. The result is a proposal, never a verdict: no matching check does not mean the change is safe. Resolve needsInput, then record the baseline with verify_change from the version before the change.

Inputs: base, runs, task, force, +3 more.

run_recovery_probeRun a Recovery Probe commandA
Destructive

Runs a Recovery Probe command that has no dedicated tool, such as check and discover for an Electron app or an attached browser, action, observe (with --steps, --network or --files), refresh, reads, windows, android --list or doctor (setup checks), and returns its exit code and output. Commands: check, discover, action, reads, windows, observe, refresh, android, doctor. Pass --json where the command supports it. See the package docs for each command's configuration.

Inputs: args (required), timeoutMs.

self_check_scenariosCheck registered scenarios before relying on themA
Destructive

Experimental. Checks scenarios from the project registry (recovery-probe.registry.json) before they are used for before/after verification. Without run it only looks: the command line of each check as its command parses it, its files, unfilled placeholders, and its configuration against the validators its command uses; nothing is started or contacted. With run true it runs each scenario that passes against the application running now, which the user declares as the reference build in reference; never pick that yourself. Each run injects the faults and performs the actions its configuration names. Each scenario ends configuration-error, execution-failed, incomplete, inconsistent, findings (on the reference build) or clean. invalidEntries lists the registry entries it could not check, and passed is false while one of them concerns the selection. Clean never means the app is correct, the scenario detects its defect, or a change is verified; report outcomes and guidance as they are.

Inputs: run, runs, registry, reference, +1 more.

set_network_faultsArm or remove network faultsA

Replaces the armed rules, or removes them all with restore: true. Windows count from this call. Set dropOpen on a rule so clients that keep connections open have to reconnect.

Inputs: rules, restore.

start_network_faultsStart a network fault proxyA
Idempotent

Starts a loopback HTTPS proxy for an app's own connections (an SDK, a main process, a CLI or a test run). Start the app or tests with the returned environment; connections pass until set_network_faults arms rules. Faults apply to whole connections per host: HTTPS stays end to end.

Inputs: port.

stop_network_faultsStop the network fault proxyA

Removes every fault, closes the proxy and returns its final summary.

Inputs: no named parameters.

verify_changeVerify a code change with before and after evidenceA

Call with baseline true before changing the code, and without it before saying the change is done. The plan file lists the checks that prove the change. Verify reruns them, compares the runs with the baseline, and scans the lines changed since the plan base for suspects. VERIFIED needs every check improved or clean with complete runs on both sides, and no suspect on a changed line; otherwise the reasons say what to resolve. Do not call a change done while it is NOT_VERIFIED.

Inputs: plan, baseline.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4/5.0

    Scored across 12 tools

    Disambiguation4/5

    The network fault lifecycle (start/set/stop/summary) and the verification workflow (plan_verification, verify_change, compare_reports) are clearly separated by explicit sequencing hints. The main soft spot is compare_reports versus verify_change, which both compare before/after runs of the same checks, and run_recovery_probe overlaps the dedicated commands it serves as a fallback for — though its description scopes it to commands without a dedicated tool.

    Naming Consistency4/5

    All names are snake_case and nearly all follow a verb_noun shape (plan_verification, set_network_faults, find_suspects, compare_reports, verify_change). The one deviation is network_fault_summary, a noun phrase where the pattern would predict something like get_network_fault_summary, which slightly weakens predictability when scanning the list.

    Tool Count5/5

    Twelve tools sits comfortably in the well-scoped range, and each maps to a distinct stage of the recovery-testing workflow rather than being filler. The single escape-hatch tool keeps the count from ballooning to cover every CLI subcommand individually.

    Completeness4/5

    The surface covers the full loop: propose checks, validate scenarios, find faults, exercise them, gather reports, compare, and gate the change. Gaps are minor — there is no tool to list or author registry/scenario entries (only self_check_scenarios and invalidEntries reporting), and no explicit report cleanup, though run_recovery_probe provides a workaround for uncovered commands.

    Maintenance

    ActivityActive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      A local MCP server that breaks on demand, allowing you to test your client against auth failures, disappearing tools, flaky responses, and token expiry from a web UI.
      10 npm
      10
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      A local-first, deterministic, read-only MCP server that audits test suites for false-green tests, tautological assertions, and mock-contract drift, ensuring tests truly validate production code. It provides tools to detect test fidelity issues, verify mock drift, and synthesize strict mock contracts.
      MIT