Skip to main content
Glama
chaoslab.server

chaoslab.server

by Ishaan1342

Agent Chaos Lab

Python 3.11+ · MIT · Experimental

An offline, stateful booking simulator for testing agent recovery policies. The key case: a booking succeeds, its response is lost, and a retry creates a second booking unless the client keeps the same idempotency key.

Tests

Install from this repository

python -m venv .venv
. .venv/bin/activate
python -m pip install -e .

See Contributing, Security and the Changelog.

Related MCP server: Travel MCP

Run the demo

Python 3.11+; the core has no third-party dependencies.

python -m chaoslab demo --output artifacts/demo.json
python -m unittest discover -s tests -v

The demo runs seven reference-policy scenarios and an intentionally broken negative control. The reference is deterministic Python, not an evaluated model. It tests clean booking, pre-commit timeout, post-commit timeout, rate limiting, price changes, a slot taken after quoting, and a lost cancellation response. Every scenario emits booking state, a tool-call trace and invariant results.

Connect an actual agent through MCP

python -m venv .venv
.venv/bin/pip install -e '.[mcp]'
CHAOS_SCENARIO=timeout_after_commit .venv/bin/python -m chaoslab.server

Use that command as a stdio MCP server in your host. Set the working directory to this repository or install the package. mcp==2.3.0 is the tested SDK. It owns MCP framing and schemas. All tools explicitly return structured output. Business faults are {ok:false,error:{code,retryable,may_have_committed}}, not transport failures. Success is {ok:true,result:...}. snapshot is an inspection tool exposing the simulator state, not something a production booking service should expose. A new subprocess creates a fresh isolated world.

Tools: quote, book, get_booking, cancel, snapshot. The test slot is court-1-1900, the original price is 4000 EUR cents. Hold one idempotency key per user intent across retries. The price-change scenario requires confirmation of a new quote; it never authorizes a higher price automatically.

.venv/bin/python -m unittest discover -s tests -v

This includes a real subprocess stdio integration test. Without the SDK that one test is explicitly skipped. An alternative serve-jsonl --scenario ... adapter accepts one {tool,arguments} object per line for non-MCP harnesses; that adapter is NOT the MCP protocol.

Scope and relationship to existing tools

Cisco's mcptoolkit-mock already provides mocking, recording and replay. This repository is a complementary stateful domain fixture, not a replacement for those capabilities and not an upstream patch. Its stateful-response request motivates the use case. A Cisco fixture/export adapter is future work.

Faults are simulated protocol outcomes, not measured network latency, a real proxy, or a benchmark of a particular model. State is in memory; no persistence or actual bookings. Multiple bookings are deliberately permitted so naive retries remain observable. No background service or model API is needed.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      A local MCP server that breaks on demand, allowing you to test your client against auth failures, disappearing tools, flaky responses, and token expiry from a web UI.
      20 npm
      10
      MIT
    • F
      license
      Not graded
      quality
      D
      maintenance
      Provides mock travel planning tools including flight and hotel search, recommendations, and multi-turn trip drafting. Supports stdio and HTTP transports and is intended for testing/demo purposes only.
      -
    • A
      license
      Not graded
      quality
      B
      maintenance
      This MCP server provides a stateful, resettable, verifiable API runtime that gates every tool call, enabling agents to run long workflows against provider-shaped environments without live provider write access. It records decisions, side effects, and outcome evidence for replayable, verifiable benchmark runs.
      Apache 2.0
    • A
      license
      B
      quality
      C
      maintenance
      Exposes availability and appointment-proposal tools over stdio so an MCP client can query open slots and suggest bookings, without being able to create them—actual booking requires human confirmation through the separate UI/API. Slot claims are transactional and idempotent, and cancellation only releases a slot still owned by that booking.
      2
      MIT