Skip to main content
Glama
README.md
# Recovery Probe

Stop assuming your code works. Test what actually happens.

Recovery Probe helps developers and AI coding agents discover bugs that ordinary tests can miss. It deliberately triggers failures in web and Electron applications, observes how the UI responds, and produces evidence you can use to reproduce, fix, and verify problems.

A failed request should not silently turn a populated screen into “No items found.” Recovery Probe helps you reproduce that failure and check whether the application recovers as expected.

[![CI](https://github.com/BojanKovachki/recovery-probe/actions/workflows/validate.yml/badge.svg)](https://github.com/BojanKovachki/recovery-probe/actions/workflows/validate.yml)
[![npm](https://img.shields.io/npm/v/recovery-probe.svg)](https://www.npmjs.com/package/recovery-probe)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

![Recovery Probe catches a broken retry flow and passes the fixed one](docs/demo.svg)

## Try it

You need Node.js 22 or newer.

```bash
npm install --save-dev recovery-probe playwright
npx playwright install chromium
npx recovery-probe demo
```

Version `1.1.0` adds an experimental verification planner, row progress tracking, and expanded runtime observation. Install the stable release with the command above.

The next preview, `1.2.0-preview.1`, adds an experimental scenario self-check: it validates registered scenarios statically and, on request, runs them a few times on a build you name, before you rely on them to verify a change. After publication, install it with `npm install --save-dev recovery-probe@next playwright`. Until then, use the [source installation instructions](CONTRIBUTING.md#install-an-unpublished-source-preview).

### Plan checks for a code change

Register existing scenarios in `recovery-probe.registry.json`, then run:

```bash
npx recovery-probe plan --changed-since main --write .recovery-probe/verify.json
```

The experimental planner selects registered checks, explains why, and lists missing coverage or expectations. It does not invent tests or declare a change correct. Review the plan, record a baseline against the before-change application, then use `verify` after the change. See [planning and registry setup](docs/plan.md). Coding agents can use the matching `plan_verification` MCP tool.

The demo runs two versions of a small app. Both load fine. Only the fixed one survives a 503, a dropped connection and a broken JSON response.

## Check your own app

**Web app.** Start it, then run:

```bash
npx recovery-probe init http://localhost:3000
```

A browser opens. Sign in, go to the screen you care about, and click something that only shows up once its data has loaded. Recovery Probe finds the request behind it, fails it three ways, and tells you whether the data came back.

**Electron app.** Start it with `--remote-debugging-address=127.0.0.1 --remote-debugging-port=9222`, then run `npx recovery-probe init`.

**Electron app that loads data over IPC.** Load the small development-only hook in the main process. `npx recovery-probe doctor` prints the code and the path to load it from. Restart the app with the inspector on, then run `npx recovery-probe init`: it lists the hook's channels, asks which one loads the screen, and fails it in a few ways.

Each run saves an HTML and JSON report. Run the same command again to repeat the check, or add `--fresh` to set it up again. If something's missing, `npx recovery-probe doctor` says what and how to fix it.

## More

| To | Run | Docs |
| --- | --- | --- |
| Fail the read behind a screen without writing assertions | `discover` | [Discovery](docs/discovery.md) |
| Catch stale screens and rows stuck in an old state, without faults | `observe` | [Observe](docs/observe.md) |
| See what users get when Save or Send fails | `action` | [Action check](docs/ipc.md#action-check) |
| Fail an SDK's or a main process's own HTTPS connections | `net` | [Network faults](docs/net.md) |
| Find code that turns a failed read into "no data" | `suspects` | [Suspects](docs/suspects.md) |
| Choose the checks a change needs (experimental) | `plan` | [Plan](docs/plan.md) |
| Check registered scenarios before relying on them (experimental) | `self-check` | [Self-check](docs/self-check.md) |
| Prove a change fixed it | `compare`, `watch`, `verify` | [Compare](docs/compare.md), [Watch](docs/watch.md), [Verify](docs/verify.md) |

## Use it from a coding agent

`recovery-probe mcp` is an MCP server. Any agent that supports MCP can use it to fail a read and look at the screen before it claims the code handles errors. Most agents take this config:

```json
{
  "mcpServers": {
    "recovery-probe": { "command": "npx", "args": ["recovery-probe", "mcp"] }
  }
}
```

Other config formats and the list of tools are in [MCP](docs/mcp.md).

## Use it in your tests

`withFault` fails matching requests inside a Playwright test:

```js
import { test, expect } from '@playwright/test'
import { withFault } from 'recovery-probe'

test('profile recovers after a 503', async ({ page }) => {
  const stats = await withFault(
    page,
    { match: '**/api/profile', kind: 'http-error', status: 503 },
    async () => {
      await page.goto('http://127.0.0.1:3000/profile')
      await page.getByRole('button', { name: 'Retry' }).click()
      await expect(page.getByTestId('profile-name')).toHaveText('Test User')
    },
  )
  expect(stats.applied).toBe(1)
})
```

If nothing matches, `withFault` throws `FAULT_NOT_TRIGGERED` instead of passing. For code that takes a `fetch` function there's `createFaultFetch`. See [API](docs/api.md).

## How it decides

- Every faulted run has a healthy run right before it, on the same screen, and the report compares the two.
- A result is a pass, a failure or inconclusive. A fault that never fired, an unstable baseline or a path it can't reach is inconclusive, never a pass.
- Some findings, like `SILENT_EMPTY`, are leads. They tell you where to look, not that the code is wrong.
- Exit codes: 0 passed, 1 a check failed, 2 needs review or inconclusive, 3 wrong usage or config, 4 couldn't run.

## Where it works

- Web apps in Chromium, either launched by Recovery Probe or attached through a local debugging port.
- Electron apps, over HTTP in the renderer or over IPC with the hook.
- Native HTTPS clients that use `HTTPS_PROXY`, such as an SDK in the main process. Only whole connections can fail there.
- Android WebViews, experimental.

It doesn't simulate a whole device going offline, WebSocket drops, token expiry, sleep or slow networks.

## Privacy

Reports stay on your machine and there's no telemetry. Request and response bodies aren't recorded, and discovery and observation keep screen text only as hashes. Reports can still contain endpoint paths, selectors, text you picked as a marker and, with `--screenshots`, what was on screen, so check them before you share them. Nothing calls an AI service except the opt-in [repair](docs/repair.md) workflow.

Failed recovery checks can also save `screenText`: up to 200 characters of newly visible screen text, included in terminal output, reports, fix briefs and MCP results. These quotes are not redacted and can contain personal or sensitive data even without screenshots. Review these outputs before sharing them or passing them to an agent.

Only test apps and accounts you're allowed to, and keep debugging ports on localhost.

## Compatibility in 1.x

The compatibility commitment covers documented, non-experimental package exports, CLI commands and options, exit codes, configuration fields, MCP tools and JSON report fields. Removing or renaming these interfaces, changing their types incompatibly, or changing their documented meaning requires a major version. Backward-compatible additions can ship in minor releases; fixes that restore documented behavior can ship in patch releases.

Consumers should tolerate additional JSON fields. Exact terminal wording, HTML layout, timing measurements and heuristic findings are not fixed contracts. Internal files under `dist/` are not public imports; use the package's declared exports. Features explicitly labeled experimental, including Android WebView support and repair, may change in minor releases, with changes recorded in the changelog.

## Upgrading to 1.0

1.0 removes the old command spellings `desktop`, `web`, `ipc`, `--demo` and `--config`. Each now stops with a usage error that names the commands that replaced it, such as `windows`, `reads`, `check`, `discover`, `action` and `demo`. Usage errors exit with 3 and setup failures with 4, where the old spellings exited with 2, so update CI scripts that read exit codes. `demo` and the JSON runner's `check` write `report.json` into the `--out` folder. Restart the development app after upgrading so its IPC hook matches the CLI. Public package imports and the hook's path are unchanged. See the [changelog](CHANGELOG.md).

## Docs

[Guided start](docs/start.md) · [Desktop and browsers](docs/desktop.md) · [IPC](docs/ipc.md) · [Discovery](docs/discovery.md) · [Observe](docs/observe.md) · [Refresh](docs/refresh.md) · [Network faults](docs/net.md) · [Android](docs/android.md) · [Suspects](docs/suspects.md) · [Compare](docs/compare.md) · [Watch](docs/watch.md) · [Plan](docs/plan.md) · [Self-check](docs/self-check.md) · [Verify](docs/verify.md) · [MCP](docs/mcp.md) · [API](docs/api.md) · [JSON runner](docs/runner.md) · [CI](docs/ci.md) · [Repair](docs/repair.md) · [Changelog](CHANGELOG.md)

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). Report security issues as described in [SECURITY.md](SECURITY.md).

## License

[MIT](LICENSE) © 2026 Bojan Kovachki

TDQS

A4/5.0

Scored across 12 tools

Disambiguation4/5

The network fault lifecycle (start/set/stop/summary) and the verification workflow (plan_verification, verify_change, compare_reports) are clearly separated by explicit sequencing hints. The main soft spot is compare_reports versus verify_change, which both compare before/after runs of the same checks, and run_recovery_probe overlaps the dedicated commands it serves as a fallback for — though its description scopes it to commands without a dedicated tool.

Naming Consistency4/5

All names are snake_case and nearly all follow a verb_noun shape (plan_verification, set_network_faults, find_suspects, compare_reports, verify_change). The one deviation is network_fault_summary, a noun phrase where the pattern would predict something like get_network_fault_summary, which slightly weakens predictability when scanning the list.

Tool Count5/5

Twelve tools sits comfortably in the well-scoped range, and each maps to a distinct stage of the recovery-testing workflow rather than being filler. The single escape-hatch tool keeps the count from ballooning to cover every CLI subcommand individually.

Completeness4/5

The surface covers the full loop: propose checks, validate scenarios, find faults, exercise them, gather reports, compare, and gate the change. Gaps are minor — there is no tool to list or author registry/scenario entries (only self_check_scenarios and invalidEntries reporting), and no explicit report cleanup, though run_recovery_probe provides a workaround for uncovered commands.

Maintenance

ActivityActive
ResponsivenessNo recent community issues