MCP Incident Lab

A standalone TypeScript MCP server for investigating synthetic payment incidents. It does not grant an agent a database, a shell, or an unscoped log dump. It never calls a paid LLM API.

The data is fixture-backed. Nothing here is a live payment system.

Why this exists

Agents are useful during incidents only if their tools are narrower than the blast radius of a mistake. This project makes that boundary testable: the model can list incidents, inspect one payment, read bounded logs, and retry a failed job after an explicit confirmation. It cannot invent SQL, grep the whole disk, or retry a job that is not failed.

flowchart LR
  Agent -->|"MCP tools"| Server
  Server --> Fixtures["Seeded incidents"]
  Server --> Audit["Audit trail"]
  Agent -.->|"cannot"| SQL["Arbitrary SQL"]
  Agent -.->|"cannot"| Shell["Shell / filesystem"]
  Agent -.->|"cannot"| Unscoped["Unscoped log search"]

Related MCP server: Payment Reconciliation Copilot

What it demonstrates

• Three seeded scenarios: a failed job, a duplicate client request, and an unhealthy dependency.

• Read-only tools: list_incidents, get_payment, list_job_attempts, search_logs.

• One mutating tool: retry_job, which requires confirm: true, a written reason, and always writes an audit event.

• Deterministic tests that diagnose each scenario from tool output alone.

Trust boundary

Read-only is the default. retry_job is the exception, and it is gated.

Likely agent failure modes

This server does not make the agent correct. It only limits the damage.

• The agent can still retry the wrong failed job.

• The agent can treat a duplicate request as a failed charge unless it reads the logs. The tools will refuse that retry because the job succeeded.

• The agent can try to “fix” an unhealthy dependency by retrying. The tools refuse that too: the job is pending, not failed, and Redis is down.

Those refusals are the point. The audit trail records accepted and rejected retries.

Quick start

npm install
npm test
npm run demo

npm run demo walks all three scenarios without an LLM.

To inspect the tools in a UI:

npm run inspector

Connect, then call list_incidents.

Cursor

Add this to MCP settings, with the project path substituted:

{
  "mcpServers": {
    "incident-lab": {
      "command": "npx",
      "args": ["tsx", "src/server.ts"],
      "cwd": "/absolute/path/to/mcp-incident-lab"
    }
  }
}

Log to stderr only. stdout is the MCP protocol.

Seeded scenarios

1. Failed job (inc_job_failed): payment still pending, three processor timeouts, retries exhausted. retry_job is accepted.

2. Duplicate request (inc_duplicate): one processed payment, logs show the second request was suppressed. retry_job is rejected.

3. Unhealthy dependency (inc_unhealthy_dep): Redis connection refused, job still pending with zero attempts. retry_job is rejected.

Failure model and tradeoffs

This is a teaching lab, not an on-call platform. Fixtures are in-memory and reset per process. There is no authentication, no real log store, and no adapter to Payment Reliability Lab in the MVP.

A production incident MCP would still keep the same shape: named lookups, hard caps, an explicit mutate-with-reason path, and an audit log. It would not expose a generic query tool and hope the model is careful.

Development

npm test
npm run build
npm run dev