Skip to main content
Glama

MCP Failure Lab

Reproduce MCP timeouts, cancellation races, transport loss, and invalid responses with repeatable tests and CI reports.

npm version CI MCP Registry GitHub MCP Registry

License: MIT

Documentation · Compatibility · Project page

Example output (duration varies):

$ npx mcp-failure-lab demo
MCP Failure Lab — Demo
Running a real 500ms delay scenario...

Scenario: Deterministic delay demo
Outcome: success
Duration: ~500 ms
Assertions: passed

Why this exists

Real MCP clients behave differently when things break. A timeout may leave a connection usable; an interrupted response may close it. A malformed reply may be rejected by one client and accepted by another.

Failure Lab makes those cases repeatable so you can check both the failed call and what happens next.

Related MCP server: mcp-chaos-rig

Real-world findings

  • Python SDK #3522: Python mcp 2.2.0 stayed closed after an interrupted HTTP response, rejected the next request, and raised an ExceptionGroup during cleanup.

  • Rust SDK #1283: rmcp 3.4.0 accepted an invalid response containing both result and error over stdio and HTTP.

  • Ruby SDK #589: mcp 1.6.1 accepted responses missing jsonrpc or declaring jsonrpc: "1.0" over stdio and HTTP in all three repeats. The Ruby report records all 60 executions; the results and upstream finding are included in the Observatory export.

  • Java SDK 2.0.1 accepted jsonrpc: "1.0" over both transports (#1156). Over stdio, responses missing jsonrpc or containing both result and error left the next ping timing out (#1157). See the Java report.

  • Duplicate-response comparison: all five tested SDKs completed the next ping. TypeScript reported the duplicate through its error callback; the other harnesses surfaced no call-level duplicate error.

The SDK comparisons describe specific tested versions, not every release. See the versioned reports for reproduction steps and limitations.

MCP Failure Lab demonstrating a bounded delay and an expected timeout

Quick start

Requires Node.js 22.23.4 or newer and npm.

npx mcp-failure-lab demo

The demo needs no API key, external server, or global installation. To test your own MCP client, connect it to Failure Lab over stdio or local Streamable HTTP:

npx mcp-failure-lab serve
# Or:
npx mcp-failure-lab serve --transport http

The stdio process waits for a client; it is not an interactive terminal command. Configure your client to launch it, or follow the getting started guide. Press Ctrl+C to stop a manually started server.

HTTP defaults to http://127.0.0.1:3000/mcp. It provides neither authentication nor TLS; do not expose it to an untrusted network.

Test your own MCP server

External targets support HTTP and stdio. After the repository setup, install the official GitHub MCP server with its executable on PATH and set GITHUB_PERSONAL_ACCESS_TOKEN in your environment. Then run:

npm run dev -- run examples/scenarios/github-get-me.json \
  --target examples/targets/github-stdio.json

The example calls GitHub's read-only get_me tool. Its target configuration uses envFrom to pass the token from your environment rather than storing it in JSON. For your own scenario and target configuration, use:

npx mcp-failure-lab run scenario.json --target target.json

External runs check tool results, deadlines, and adapter setup and cleanup. They do not inject faults into another server; Failure Lab is not a proxy.

See the external targets guide for prerequisites, credential handling, and configuration.

What Failure Lab can break

The built-in server exposes these tools for testing client behavior:

Tool

Behavior

ping

Returns a deterministic health response

protocol_ping_liveness

Sends a bounded protocol ping during its in-flight call

delay

Waits for a bounded duration before returning

hang

Remains pending until the client cancels

disconnect

Interrupts the active transport while a request is in flight

malformed_message

Violates one selected JSON-RPC response rule exactly once

duplicate_response

Sends the same JSON-RPC response twice for one request

response_after_cancellation

Sends one late response for a cancelled stdio request

session_loss

Invalidates the caller's legacy HTTP session

oversized_response

Returns a deterministic text payload up to 256 KiB

invalid_progress

Sends an unknown, regressing, or late progress notification

Protocol ping is distinct from the ping tool. Built-in scenarios default to MCP 2026-07-28; protocol liveness requires scenario protocolVersion: "2025-11-25". Both public transports accept legacy clients.

See the fault tools reference for timing bounds, activation, cancellation, cleanup, and transport limits.

oversized_response accepts sizeBytes from 0 through 262,144. It generates only ASCII x characters, so the text size is exact in UTF-8 bytes and contains no external data. The cap applies on both stdio and HTTP. Choose a size below your client's configured response limit to check success, then a size above that limit (but no more than 262,144) to check client rejection. The included 64 KiB scenario checks the exact returned text size and a follow-up request. The JSON-RPC and transport framing add bytes around the text, so a client's whole-message limit may trigger before sizeBytes reaches its threshold. The integration test confirms that a stdio client with a 4 KiB message limit accepts a 1 KiB payload and rejects an 8 KiB payload.

invalid_progress requires a client progress token and accepts unknown-token, regressing-progress, or after-completion. These are protocol notifications associated with one tool call; they are not tool result content. The unknown-token variant sends one notification with a different token. The regressing variant sends progress 2 followed by 1. The late variant sends one notification after the matching response on stdio or in the HTTP response stream. These violate request-token correlation, monotonic progress, and the completed-request lifecycle, respectively. Each variant is bounded and consumed once. See the regressing, unknown-token, and after-completion scenarios. The built-in runner can assert callback values with expect.progressValues; external target adapters do not report progress notifications and cannot use that assertion.

How recovery testing works

Trigger a fault, record its outcome, then make a second request on the same connection. A detected fault does not prove recovery; the follow-up must succeed too.

Scenario observe calls run after the primary call, including errors and timeouts. For example, duplicate-response.json calls duplicate_response, then uses ping to check that the connection remains usable:

# From a repository checkout
npm run dev -- run examples/scenarios/duplicate-response.json

This checks post-fault behavior, not an automatic recovery policy. A stdio disconnect terminates the server process and requires a new process and connection.

See Scenarios for outcome, duration, result, and observer assertions.

SDK / transport evidence

MCP Failure Observatory collects SDK and transport evidence. The repository reports preserve tested versions, methods, and limitations.

Results vary by SDK, version, transport, and protocol. A passing run is evidence for that combination, not a guarantee for every client.

CI usage

Save a scenario file with explicit expectations and timeouts, then produce a JUnit report:

npx mcp-failure-lab run scenario.json --report junit > junit.xml

Pin the Failure Lab package version in CI so upgrades do not change the test environment unexpectedly.

Exit codes: 0 means expectations passed, 1 means the scenario could not be loaded or executed, and 2 means an assertion failed. An expected timeout can pass; an unexpected success can fail.

See Reporting for JSON, JUnit, and lifecycle diagnostics.

Documentation

Detailed guides live at mcplab.dev: Getting started · CLI · Architecture · Troubleshooting.

Contributing

See CONTRIBUTING.md for setup, tests, and the contribution workflow. Planned work is tracked in GitHub Issues.

If Failure Lab helps you test an MCP integration, consider starring the repository.

License

MIT

Available Tools

10 tools
delayA

Delay a successful MCP response to test client timeout behavior.

Inputs: delayMs (required).

disconnectA

Close the MCP transport before this request can receive a response.

Inputs: no named parameters.

duplicate_responseB

Return the same JSON-RPC response twice for this request.

Inputs: no named parameters.

hangB

Never respond unless the MCP request is cancelled.

Inputs: no named parameters.

invalid_progressA

Send one bounded progress-notification fault for this request; requires a client progress token.

Inputs: variant (required).

malformed_messageC

Return one deterministic response that violates a selected JSON-RPC rule.

Inputs: variant (required).

oversized_responseA

Return a deterministic text payload of exactly sizeBytes UTF-8 bytes (maximum 256 KiB).

Inputs: sizeBytes (required).

pingB

Check whether the MCP server is responsive.

Inputs: no named parameters.

protocol_ping_livenessC

Send an MCP protocol ping during this in-flight call and apply a bounded liveness policy.

Inputs: pingAfterMs (required), livenessTimeoutMs (required), closeOnFailure, completionDelayMs.

response_after_cancellationA

Send a response after this call is cancelled (stdio only).

Inputs: no named parameters.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.3/5.0

    Scored across 10 tools

    Disambiguation4/5

    Most tools target clearly distinct failure modes such as delay, disconnect, hang, malformed_message, and duplicate_response. However, ping and protocol_ping_liveness both concern liveness checking and could be confused, keeping this from a perfect score.

    Naming Consistency4/5

    All tool names use consistent snake_case and are descriptive. The main deviation is grammatical style: some are bare verbs (delay, disconnect, hang, ping) while others are noun phrases (oversized_response, malformed_message).

    Tool Count5/5

    Ten tools is well-scoped for a failure-injection lab. Each tool represents a distinct test scenario and none feels redundant or excessive.

    Completeness4/5

    The set covers core transport, protocol, timing, and cancellation failure modes, including oversized responses, malformed messages, duplicates, and post-cancellation responses. It lacks explicit injection of JSON-RPC error responses or invalid request IDs, which are common client-handling tests.

    Maintenance

    ActivityActive
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      A comprehensive reference implementation demonstrating all features of the Model Context Protocol (MCP) specification, serving as documentation, learning resource, and testing tool for MCP implementations.
      1
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      A local MCP server that breaks on demand, allowing you to test your client against auth failures, disappearing tools, flaky responses, and token expiry from a web UI.
      10 npm
      10
      MIT