Skip to main content
Glama

Recovery Probe

Stop assuming your code works. Test what actually happens.

Recovery Probe helps developers and AI coding agents discover bugs that ordinary tests can miss. It deliberately triggers failures in web and Electron applications, observes how the UI responds, and produces evidence you can use to reproduce, fix, and verify problems.

A failed request should not silently turn a populated screen into “No items found.” Recovery Probe helps you reproduce that failure and check whether the application recovers as expected.

CI npm License: MIT

Recovery Probe catches a broken retry flow and passes the fixed one

Try it

You need Node.js 22 or newer.

npm install --save-dev recovery-probe playwright
npx playwright install chromium
npx recovery-probe demo

The demo runs two versions of a small app. Both load fine. Only the fixed one survives a 503, a dropped connection and a broken JSON response.

Related MCP server: mcp-chaos-rig

Check your own app

Web app. Start it, then run:

npx recovery-probe init http://localhost:3000

A browser opens. Sign in, go to the screen you care about, and click something that only shows up once its data has loaded. Recovery Probe finds the request behind it, fails it three ways, and tells you whether the data came back.

Electron app. Start it with --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222, then run npx recovery-probe init.

Electron app that loads data over IPC. Load the small development-only hook in the main process. npx recovery-probe doctor prints the code and the path to load it from. Restart the app with the inspector on, then run npx recovery-probe init: it lists the hook's channels, asks which one loads the screen, and fails it in a few ways.

Each run saves an HTML and JSON report. Run the same command again to repeat the check, or add --fresh to set it up again. If something's missing, npx recovery-probe doctor says what and how to fix it.

More

To

Run

Docs

Fail the read behind a screen without writing assertions

discover

Discovery

Catch stale screens and rows stuck in an old state, without faults

observe

Observe

See what users get when Save or Send fails

action

Action check

Fail an SDK's or a main process's own HTTPS connections

net

Network faults

Find code that turns a failed read into "no data"

suspects

Suspects

Prove a change fixed it

compare, watch, verify

Compare, Watch, Verify

Use it from a coding agent

recovery-probe mcp is an MCP server. Any agent that supports MCP can use it to fail a read and look at the screen before it claims the code handles errors. Most agents take this config:

{
  "mcpServers": {
    "recovery-probe": { "command": "npx", "args": ["recovery-probe", "mcp"] }
  }
}

Other config formats and the list of tools are in MCP.

Use it in your tests

withFault fails matching requests inside a Playwright test:

import { test, expect } from '@playwright/test'
import { withFault } from 'recovery-probe'

test('profile recovers after a 503', async ({ page }) => {
  const stats = await withFault(
    page,
    { match: '**/api/profile', kind: 'http-error', status: 503 },
    async () => {
      await page.goto('http://127.0.0.1:3000/profile')
      await page.getByRole('button', { name: 'Retry' }).click()
      await expect(page.getByTestId('profile-name')).toHaveText('Test User')
    },
  )
  expect(stats.applied).toBe(1)
})

If nothing matches, withFault throws FAULT_NOT_TRIGGERED instead of passing. For code that takes a fetch function there's createFaultFetch. See API.

How it decides

  • Every faulted run has a healthy run right before it, on the same screen, and the report compares the two.

  • A result is a pass, a failure or inconclusive. A fault that never fired, an unstable baseline or a path it can't reach is inconclusive, never a pass.

  • Some findings, like SILENT_EMPTY, are leads. They tell you where to look, not that the code is wrong.

  • Exit codes: 0 passed, 1 a check failed, 2 needs review or inconclusive, 3 wrong usage or config, 4 couldn't run.

Where it works

  • Web apps in Chromium, either launched by Recovery Probe or attached through a local debugging port.

  • Electron apps, over HTTP in the renderer or over IPC with the hook.

  • Native HTTPS clients that use HTTPS_PROXY, such as an SDK in the main process. Only whole connections can fail there.

  • Android WebViews, experimental.

It doesn't simulate a whole device going offline, WebSocket drops, token expiry, sleep or slow networks.

Privacy

Reports stay on your machine and there's no telemetry. Request and response bodies aren't recorded, and discovery and observation keep screen text only as hashes. Reports can still contain endpoint paths, selectors, text you picked as a marker and, with --screenshots, what was on screen, so check them before you share them. Nothing calls an AI service except the opt-in repair workflow.

Failed recovery checks can also save screenText: up to 200 characters of newly visible screen text, included in terminal output, reports, fix briefs and MCP results. These quotes are not redacted and can contain personal or sensitive data even without screenshots. Review these outputs before sharing them or passing them to an agent.

Only test apps and accounts you're allowed to, and keep debugging ports on localhost.

Compatibility in 1.x

The compatibility commitment covers documented, non-experimental package exports, CLI commands and options, exit codes, configuration fields, MCP tools and JSON report fields. Removing or renaming these interfaces, changing their types incompatibly, or changing their documented meaning requires a major version. Backward-compatible additions can ship in minor releases; fixes that restore documented behavior can ship in patch releases.

Consumers should tolerate additional JSON fields. Exact terminal wording, HTML layout, timing measurements and heuristic findings are not fixed contracts. Internal files under dist/ are not public imports; use the package's declared exports. Features explicitly labeled experimental, including Android WebView support and repair, may change in minor releases, with changes recorded in the changelog.

Upgrading to 1.0

1.0 removes the old command spellings desktop, web, ipc, --demo and --config. Each now stops with a usage error that names the commands that replaced it, such as windows, reads, check, discover, action and demo. Usage errors exit with 3 and setup failures with 4, where the old spellings exited with 2, so update CI scripts that read exit codes. demo and the JSON runner's check write report.json into the --out folder. Restart the development app after upgrading so its IPC hook matches the CLI. Public package imports and the hook's path are unchanged. See the changelog.

Docs

Guided start · Desktop and browsers · IPC · Discovery · Observe · Refresh · Network faults · Android · Suspects · Compare · Watch · Verify · MCP · API · JSON runner · CI · Repair · Changelog

Contributing

See CONTRIBUTING.md. Report security issues as described in SECURITY.md.

License

MIT © 2026 Bojan Kovachki

Available Tools

10 tools
check_read_recoveryCheck that a page recovers when a read failsA

Loads the page healthy, then fails one GET endpoint with each fault kind and checks that the ready marker comes back: by itself (recovery "automatic") or after clicking retrySelector ("retry"). Returns an outcome per fault, why the healthy baseline failed if it did, the text the screen showed under a fault it did not recover from, and the report folder, which includes fix-brief.md and reproduce.mjs.

NameRequiredDescriptionDefault
urlYesPage of a development or test app.
timesNoHow many consecutive requests each fault fails.
faultsNoWhich faults to apply; all of http-error, connection-failure, invalid-json by default.
endpointYesOrigin and path of the GET read to fail, as find_page_reads lists it.
recoveryNoautomatic
readyTextNoText the ready element must contain.
timeoutMsNo
storageStateNo
readySelectorYesCSS selector of the one element that is visible only once that read has rendered its data. It must match exactly one element: for a list, choose its container or a single item, not a selector every item matches.
retrySelectorNoRequired when recovery is "retry".

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations say readOnlyHint=false, destructiveHint=false, openWorldHint=true, and the description is consistent with that: it drives a real browser, injects transient network faults, and clicks a retry control. It adds meaningful context beyond the annotations, notably that artifacts are written to a report folder with fix-brief.md and reproduce.mjs, and that faults run one endpoint at a time. It stops short of stating auth/session requirements (storageState) or the side effects of leaving faults applied if the run aborts.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that leads with the action (loads healthy, then fails one GET) before the recovery logic and return payload. It is dense but every clause earns its place; the return-value enumeration is slightly overstuffed for one sentence but not wasteful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 10 parameters, no output schema, and open-world behavior, the description does a creditable job covering the return shape (outcome per fault, baseline failure reason, on-screen text, report folder). What is missing is failure/abort semantics and environment preconditions, but for a tool of this complexity it is close to sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 70%, so the description must carry some load, and it does: it explains that recovery mode selects automatic-vs-retry behavior, that retrySelector only matters in 'retry' mode, and that each fault kind is exercised against a single endpoint. It adds no meaning for timeoutMs, storageState, readyText, or times, which the schema already documents adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource with unusual precision: load healthy, fail one GET endpoint per fault kind, verify the ready marker returns either automatically or after clicking retrySelector. The one gap is sibling differentiation — run_recovery_probe and the start/stop_network_faults family are never named or contrasted, so an agent must infer that this is the end-to-end composite rather than a primitive.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the mechanics described; there is no explicit 'use this when' or 'prefer X instead' statement, and no stated prerequisites (dev/test app, browser availability) beyond a hint carried in the url parameter's schema text. The recovery enum ('automatic' vs 'retry') tells the agent it must know which recovery strategy the page implements, but not how to choose the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_reportsCompare reports from before and after a changeA
Read-only

Compares reports of the same check from before and after a change, such as main and a branch, and returns a verdict (IMPROVED, PARTLY_IMPROVED, REGRESSED, MIXED, UNCHANGED, INCONCLUSIVE or NOT_COMPARABLE), what happened to each problem, and a Markdown block for a pull request. A problem counts as gone only when a complete run after the change checked it, so a run that stopped early is never a fix. Give several reports per side for repeated runs.

NameRequiredDescriptionDefault
afterYesReports of the same check from after the change.
beforeYesReports from before the change: report.json files or their folders.
afterLabelNoName for the after side, such as the branch.
beforeLabelNoName for the before side, such as main.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint true, openWorldHint false), yet the description still adds real behavioral content: the exact verdict set, the rule that a problem only counts as gone when a complete run verified it, and that early-stopped runs are never treated as fixes. It does not discuss failure modes for invalid report paths or how verdicts are weighted, keeping it below a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences, all front-loaded: purpose and return shape first, then the correctness rule, then the input-volume hint. No sentence is redundant and nothing is buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return value and does so by naming the verdicts and the Markdown block, which is enough for an agent to know what comes back. The remaining gap is minor: it never says what happens when the sides are from different checks or how a bad path surfaces.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description does add meaning to the array parameters — it explains why multiple reports per side are allowed (repeated runs) and gives the label parameters concrete purpose ("such as main and a branch"), but it contributes no format details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a precise verb and resource ("compares reports of the same check from before and after a change") and enumerates the full verdict vocabulary plus the returned artifacts (per-problem outcome, Markdown PR block). It does not name the closest sibling (verify_change) or state how it differs, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Give several reports per side for repeated runs" is an explicit usage instruction, and the "complete run" rule tells the agent when a comparison result is trustworthy. There is no guidance on when to prefer verify_change or what to do when inputs come from a single run, so exclusions are absent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_page_readsFind the reads behind a pageA
Read-only

Opens a page in headless Chromium and lists the JSON GET requests it makes while it loads. Use it to choose the endpoint for check_read_recovery. Only successful page fetch and XHR reads are seen; native, IPC and service-worker traffic is not.

NameRequiredDescriptionDefault
urlYesPage of a development or test app.
durationMsNoHow long to watch after the page loads.
storageStateNoPlaywright storage state file of a signed-in test account.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark it read-only and open-world, and the description adds genuinely new limits: only successful page-fetch and XHR reads are captured while native, IPC and service-worker traffic is invisible. That coverage gap is the key behavioral fact an agent needs before trusting a negative result; only the return format/pagination detail is missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with zero filler, front-loaded with the mechanism and output, then the hand-off, then the limitation. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of describing results and does so ('lists the JSON GET requests'), plus a caveat about what it misses. It could say more about ordering, volume, or how signed-in state affects visibility, but an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so durationMs, url and storageState are already documented, but the description still adds meaning by tying capture to the page-load window ('while it loads'), clarifying what durationMs governs. Marginal but real value on top of the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Opens a page in headless Chromium and lists the JSON GET requests it makes while it loads') and scopes the output to JSON GET reads, which clearly separates it from siblings like compare_reports or verify_change.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit downstream use: 'Use it to choose the endpoint for check_read_recovery,' which tells the agent when this tool precedes another. There are no explicit when-not conditions or named alternatives beyond that hand-off, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_suspectsFind failure handling that can lose dataA
Read-only

Reads JavaScript and TypeScript sources, without running them, for two patterns: UNKNOWN_AS_EMPTY (a failed or unexpected read becomes an empty collection, so "unknown" looks like "none") and FAILURE_ONLY_LOGGED (a read stores its data when it succeeds, but its failure is only logged). Each suspect names the read to fail and the check that proves or dismisses it. Use changedSince "main" to scan only what a branch changed. Needs TypeScript in the project.

NameRequiredDescriptionDefault
pathsNoFiles and directories to scan, relative to the working directory. Default: all of it.
changedSinceNoGit reference, such as main: scan only the files changed since it, uncommitted and untracked ones included.
includeTestsNoScan test files too.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, but the description adds the meaningful detail that sources are analyzed statically "without running them," and it discloses the TypeScript dependency, which the annotations do not. It also sketches the response shape (each suspect names the read and the check), going beyond the structured fields without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with what is scanned and what counts as a suspect, then the workflow tip and the prerequisite. Nothing repeats the title or the schema verbatim, and every sentence contributes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by explaining what a returned suspect contains, and it covers the static-analysis behavior, the branch-diff workflow, and the TypeScript requirement. For a three-optional-parameter read-only scanner, nothing an agent needs in order to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so paths, changedSince, and includeTests are already fully documented, and the description only echoes the changedSince="main" case. Baseline 3 is appropriate; the description adds negligible semantics the schema does not already carry.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource ("Reads JavaScript and TypeScript sources, without running them") and names the two exact defect patterns it looks for, UNKNOWN_AS_EMPTY and FAILURE_ONLY_LOGGED, which makes the tool's remit unambiguous. It never names or contrasts with the sibling read-analysis tools (find_page_reads, check_read_recovery), so an agent working from names alone still has to infer the split.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use changedSince \"main\" to scan only what a branch changed" supplies a concrete usage context, and "Needs TypeScript in the project" is a real prerequisite that prevents failed invocations. There are no explicit when-not conditions or pointers to the sibling tools for adjacent questions, so the guidance is clear but not complete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

network_fault_summaryShow what the network faults didA
Read-only

Reports how many connections went through the proxy and, for the armed rules, how many each matched and faulted. A rule that faulted nothing means the app did not use the proxy or the host name is wrong.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

readOnlyHint=true already tells the agent this is a safe read with no side effects, so the bar is lower. Beyond that, the description adds genuinely useful signal: what is counted (total proxy connections, per-rule matches and faults) and how to read a zero-fault rule as a likely misconfiguration.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences. The core output description comes first, and the troubleshooting hint follows, so an agent gets the essential behavior immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must convey the return shape, and it does: a total proxy connection count plus per-armed-rule matched and faulted counts. Adding an explicit note on when to run it relative to the start/set tools would make it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no parameter semantics for the description to explain. The baseline for a parameterless tool applies cleanly, and there is nothing misleading about the omission.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: reports proxy connection counts and, per armed rule, matched and faulted counts. This clearly separates it from the sibling control tools (start/set/stop_network_faults) which configure rather than report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Mentions 'armed rules', implying this is consulted after rules are set, but never explicitly says when to call it versus siblings like compare_reports or find_suspects. The line about a rule that faulted nothing indicates a misconfigured host or unused proxy, which is useful diagnostic guidance even though it is about interpreting results rather than selecting the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_recovery_probeRun a Recovery Probe commandA
Destructive

Runs a Recovery Probe command that has no dedicated tool, such as check and discover for an Electron app or an attached browser, action, observe (with --steps, --network or --files), refresh, reads, windows, android --list or doctor (setup checks), and returns its exit code and output. Commands: check, discover, action, reads, windows, observe, refresh, android, doctor. Pass --json where the command supports it. See the package docs for each command's configuration.

NameRequiredDescriptionDefault
argsYesCommand and arguments, for example ["observe", "--config", "observe.json", "--json"].
timeoutMsNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds genuine behavior beyond the annotations by disclosing the return value ('exit code and output') and the accepted command set. However, despite destructiveHint=true and openWorldHint=true, it never warns that commands may mutate state or require setup/permissions, and 'See the package docs' defers the details an agent most needs before invoking an arbitrary-command runner.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded, but the same command list is enumerated twice — once inline with parenthetical qualifiers and again as a flat 'Commands:' list — which is pure repetition. The closing 'See the package docs' sentence mostly shifts work elsewhere rather than informing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the essentials for a generic command runner with no output schema: valid commands, output shape (exit code + output), and --json. Missing is any statement of side effects, permission/setup requirements, or what happens on failure for a tool flagged destructive and open-world, which is a meaningful omission for a catch-all runner.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema documents args only generically, but the description supplies the actual valid command vocabulary (check, discover, action, reads, windows, observe, refresh, android, doctor) plus --json guidance, which meaningfully constrains args[0] where the schema has no enum. timeoutMs is never mentioned, and 'see the package docs' leaves per-command configuration unresolved.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Runs a Recovery Probe command') and enumerates the concrete commands it accepts, so an agent knows exactly what surface it covers. The 'has no dedicated tool' phrase is a strong scoping clue that differentiates it from named siblings, though 'Recovery Probe' itself is jargon that offers no domain anchor.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear fallback rule: use this only for commands that have no dedicated tool, which is exactly the routing decision an agent needs when siblings like check_read_recovery or network_fault_summary exist. It also advises passing --json where supported, but never names a specific sibling to prefer or states what happens when a dedicated tool does exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_network_faultsArm or remove network faultsA

Replaces the armed rules, or removes them all with restore: true. Windows count from this call. Set dropOpen on a rule so clients that keep connections open have to reconnect.

NameRequiredDescriptionDefault
rulesNo
restoreNo

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safety profile (readOnlyHint=false, destructiveHint=false), and the description adds genuinely non-obvious behavior: the call replaces rather than merges existing rules, fault windows are measured from invocation time, and dropOpen forces reconnection for persistent clients. No contradiction with the annotations, though it doesn't explain whether the change is reversible or scoped to a session.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the replacement/restore contract ahead of the secondary details. Dense and jargon-heavy but every sentence carries a distinct fact; minor packing of two ideas into the dropOpen sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nested rule-array fault-injection tool with no output schema and no annotations covering side effects, the description covers the replace/restore contract, timing origin, and dropOpen. It still leaves the fault modes and count semantics unexplained, which is a meaningful gap for correctly constructing a rule.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and only the host pattern has any inline documentation, so the description must compensate. It does explain restore and dropOpen and clarifies that fromMs/untilMs windows are relative to this call, which is valuable, but the fault enum values (refuse/reset/stall/delay) and count remain undefined anywhere.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and semantics: it replaces the armed rule set, or clears it entirely via restore: true. It doesn't name the sibling tools (start_network_faults/stop_network_faults) that an agent would otherwise confuse this with, so sibling differentiation is only implied.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives conditional guidance for one branch ('removes them all with restore: true') and implies replacement semantics for the other, which is useful. But it never says when to prefer this over start_network_faults or stop_network_faults, leaving the main routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_network_faultsStart a network fault proxyA
Idempotent

Starts a loopback HTTPS proxy for an app's own connections (an SDK, a main process, a CLI or a test run). Start the app or tests with the returned environment; connections pass until set_network_faults arms rules. Faults apply to whole connections per host: HTTPS stays end to end.

NameRequiredDescriptionDefault
portNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is non-read-only, idempotent, and non-destructive. The description adds useful behavioral context beyond that: connections pass through until rules are armed, faults apply per host to whole connections, and HTTPS remains end to end. It does not describe the returned environment's structure in detail, but the key runtime behavior is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three tightly written sentences with the primary action front-loaded. Each sentence contributes useful scope, usage sequencing, or behavioral detail, and there is no redundant repetition of annotations or schema constraints.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's main behavior and its relationship to set_network_faults, which is the most important context. However, with no output schema, the returned environment is only mentioned vaguely, and the sole input parameter is not explained at all. For a startup tool with one optional parameter, those gaps leave it only minimally complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one optional parameter, port, with 0% description coverage. The description never explains what the port parameter controls, whether it has a default, or what happens if it is omitted or already in use. The schema only provides type and range constraints, so the parameter's meaning remains undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: starts a loopback HTTPS proxy for an app's own connections. It also scopes the proxy to SDKs, main processes, CLIs, or test runs, making it distinguishable from sibling tools that arm faults or stop the proxy.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: start the app or tests with the returned environment. It also routes the agent to set_network_faults for arming rules and clarifies that connections pass until then. It does not explicitly cover when not to use this tool or when to call stop_network_faults, so it falls short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_network_faultsStop the network fault proxyA

Removes every fault, closes the proxy and returns its final summary.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses three concrete side effects: all faults are removed (not selectively), the proxy is closed, and a final summary is returned. It stops short of stating idempotency behavior or what happens if no proxy is running, which matters for a teardown call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short, front-loaded sentence with the primary action first and the byproducts after. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-argument teardown with annotations covering safety, this is nearly complete: side effects and the returned summary are named. The remaining gap is that the summary's shape is unspecified and there is no output schema to fall back on.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing to disambiguate; the description correctly says the scope of removal is everything rather than a subset. Baseline for a parameterless tool applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('removes every fault, closes the proxy') and implies the teardown counterpart to start_network_faults and set_network_faults. It is clear what the tool does, though it never names a sibling tool to sharpen the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the agent must infer that this is the cleanup step after start_network_faults. There is no explicit when-to-use/when-not, no prerequisite (e.g. an active proxy), and no mention of the summary-only alternative network_fault_summary.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_changeVerify a code change with before and after evidenceA

Call with baseline true before changing the code, and without it before saying the change is done. The plan file lists the checks that prove the change. Verify reruns them, compares the runs with the baseline, and scans the lines changed since the plan base for suspects. VERIFIED needs every check improved or clean with complete runs on both sides, and no suspect on a changed line; otherwise the reasons say what to resolve. Do not call a change done while it is NOT_VERIFIED.

NameRequiredDescriptionDefault
planNoThe plan file, relative to the working directory. Default: .recovery-probe/verify.json.
baselineNoRecord the runs from before the change instead of verifying it.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare that this is not read-only and is closed-world; the description adds substantial behavior beyond that: it records baseline runs, reruns checks, compares both sides, scans changed lines, and defines the VERIFIED condition and NOT_VERIFIED gating. It stops short of describing response formatting or persistence details, but the verdict semantics are unusually well disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded: the baseline workflow comes first, then the verification mechanism, then the pass/fail rule. Most sentences carry useful information, though the phrase about scanning changed lines is slightly compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a stateful verification tool with no output schema, the description sufficiently explains prerequisites, invocation timing, what the tool does, and what determines VERIFIED versus NOT_VERIFIED. It does not document the plan file format, but that is deferred to the plan file itself.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already defines the plan path and baseline boolean. The description adds operational meaning beyond the schema by explaining when to set baseline true versus false and how the plan file drives the verification checks.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (verify), resource (a code change), and evidence model (before/after runs). It clearly positions itself against siblings like find_suspects and compare_reports by explaining that it reruns the plan's checks, compares runs with the baseline, and scans changed lines for suspects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs the agent to call with baseline true before changing code and without it before declaring the change done. It also states a clear exclusion: do not call a change done while it is NOT_VERIFIED.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 10 tool updatesv1.0.2
    • First observedcheck_read_recovery
    • First observedcompare_reports
    • First observedfind_page_reads
    • First observedfind_suspects
    • First observednetwork_fault_summary
    • First observedrun_recovery_probe
    • First observedset_network_faults
    • First observedstart_network_faults
    • First observedstop_network_faults
    • First observedverify_change

TDQS

A3.9/5.0

Scored across 10 tools

Disambiguation4/5

Most tools target clearly distinct steps in the recovery-testing workflow, and the start/set/stop network-fault trio is especially clear. There is some overlap between check_read_recovery and run_recovery_probe (which can run a generic check), and between compare_reports and verify_change, though the descriptions mostly disambiguate them.

Naming Consistency4/5

Almost all tools use a predictable snake_case verb_noun pattern, and the fault lifecycle tools are consistently named. The main deviation is network_fault_summary, which is noun-first rather than verb-led, but the set remains readable.

Tool Count5/5

Ten tools is well-scoped for a recovery probing server covering fault injection, endpoint discovery, recovery checks, static suspect scanning, report comparison, verification, and a generic command runner. Each tool appears to earn its place without an excessive surface.

Completeness4/5

The network-fault lifecycle, recovery check, suspect discovery, report comparison, and change verification workflows are well covered. Minor gaps remain around directly creating or managing the plan file used by verify_change and listing or cleaning up reports, though the generic run_recovery_probe tool helps cover other commands.

Maintenance

ActivityActive
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    C
    quality
    C
    maintenance
    Enables running visual regression tests, polling results, and reading structured failure reports via MCP clients like Cursor or Claude Desktop.
    22
    9 npm
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    A local MCP server that breaks on demand, allowing you to test your client against auth failures, disappearing tools, flaky responses, and token expiry from a web UI.
    7 npm
    10
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    A local-first, deterministic, read-only MCP server that audits test suites for false-green tests, tautological assertions, and mock-contract drift, ensuring tests truly validate production code. It provides tools to detect test fidelity issues, verify mock drift, and synthesize strict mock contracts.
    MIT