# MCP Incident Responder

An AI-native incident response server built on the **Model Context Protocol (MCP)**. Exposes diagnostic tools that any MCP-compatible LLM agent can discover and call over HTTP using standard JSON-RPC 2.0.

---

## Roadmap

| Phase | What | Status |
|-------|------|--------|
| **Phase 1** | Build MCP server + 3 tools | ✅ Done |
| **Phase 2** | Test all JSON-RPC calls in Bruno | ✅ Done |
| **Phase 3** | Connect LLM agent | Next |
| **Phase 4** | Add auth (Bearer token) | Enhancement |
| **Phase 5** | Connect real data (CloudWatch, etc.) | Enterprise extension |

---

## Phase 1 — Build the MCP Server ✅

### What we built

A Python MCP server that runs on `http://localhost:8002` and exposes three tools and one prompt template over a single HTTP endpoint: `POST /mcp`.

**Project structure:**

```
mcp-incident-responder/
├── server.py              ← MCP server entry point
├── tools/
│   ├── __init__.py        ← Registers all tools with the server
│   ├── system_status.py   ← Tool: getSystemStatus
│   ├── recent_errors.py   ← Tool: getRecentErrors
│   └── create_ticket.py   ← Tool: createTicket
├── data/
│   └── mock_data.py       ← Simulated metrics and logs
└── requirements.txt
```

---

### The 3 Tools

| Tool | Type | What it does |
|------|------|--------------|
| `getSystemStatus` | Read-only | Returns CPU, memory, uptime, and health status for all services |
| `getRecentErrors` | Read-only | Returns recent error logs, optionally filtered by service name |
| `createTicket` | Action | Creates an incident ticket in the (simulated) ticketing system |

The mock data in `data/mock_data.py` simulates a realistic degraded system:
- `auth-service` → healthy
- `payment-service` → degraded (CPU ~92%, memory ~90%, DB connection pool exhausted)
- `notification-service` → down (crashed with OutOfMemoryError)

---

### The Prompt Template

The server also exposes a prompt called `incidentAnalysisTemplate` that gives an LLM a structured workflow:

```
ASSESS → INVESTIGATE → CORRELATE → DIAGNOSE → RECOMMEND → ACT
```

An LLM agent can fetch this prompt from the server and follow it automatically — the server defines the workflow, not the client.

---

### Running the server

```bash
pip install "mcp[cli]" fastapi uvicorn
python server.py
```

Server starts on `http://localhost:8002`. Health check available at `GET /health`.

---

### Phase 1 Verification — The MCP Handshake

The first thing any MCP client does is send an `initialize` request. This is **capability discovery** — the client asks "who are you and what can you do?" before calling any tools.

**Request** (`POST http://localhost:8002/mcp`):

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {
    "protocolVersion": "2025-03-26",
    "capabilities": {
      "roots": { "listChanged": true },
      "sampling": {}
    },
    "clientInfo": {
      "name": "BrunoClient",
      "title": "Incident Responder Test",
      "version": "1.0.0"
    }
  }
}
```

**Response** (`200 OK`):

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2025-03-26",
    "capabilities": {
      "experimental": {},
      "prompts": { "listChanged": false },
      "resources": { "subscribe": false, "listChanged": false },
      "tools": { "listChanged": false }
    },
    "serverInfo": {
      "name": "Incident Responder",
      "version": "1.27.2"
    }
  }
}
```

#### What this exchange means

The client introduces itself and declares the protocol version it speaks. The server responds with its own identity (`Incident Responder`, SDK `v1.27.2`) and confirms it supports the same protocol version. It also advertises its capability categories: `tools`, `prompts`, and `resources`.

No tools were called yet — this is purely the handshake. The client now knows what's available and can start making targeted requests.

> **Phase 1 is complete.** The server is running, the tools are registered, and the MCP handshake succeeds.

---

## Phase 2 — Manual Testing with Bruno ✅

Phase 2 is about **proving every part of the server works** by manually firing each JSON-RPC request and reading the response. You are the brain here — you decide what to call next, read the output, and reason about it. In Phase 3, an LLM takes over that role automatically.

### The 6 requests, in order

| # | Method | What it does | Result |
|---|--------|--------------|--------|
| 1 | `initialize` | Handshake — discover capabilities | ✅ 200 OK |
| 2 | `tools/list` | List all registered tools | ✅ 200 OK |
| 3 | `tools/call → getSystemStatus` | Check live service health | ✅ 200 OK |
| 4 | `tools/call → getRecentErrors` | Retrieve filtered error logs | ✅ 200 OK |
| 5 | `tools/call → createTicket` | Create an incident ticket | ✅ 200 OK |
| 6 | `prompts/list` | List available prompt templates | ✅ 200 OK |

---

### Request 2 — `tools/list`

After the handshake, ask the server to enumerate every tool it has registered.

**Request:**

```json
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list",
  "params": {}
}
```

**Response** (`200 OK`) — server returns all 3 tools with their input schemas so the client knows exactly what arguments each tool accepts.

---

### Request 3 — `tools/call → getSystemStatus`

Call the first tool to get a live snapshot of all services.

**Request:**

```json
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "getSystemStatus",
    "arguments": {}
  }
}
```

**Response** — the mock data returns a degraded system:
- `auth-service` → healthy
- `payment-service` → **degraded** (CPU ~92%, memory ~90%)
- `notification-service` → **down** (port 8080 refused)
- `overall_status` → **critical**

---

### Request 4 — `tools/call → getRecentErrors`

Drill into the logs for `payment-service` to understand why it is degraded.

**Request:**

```json
{
  "jsonrpc": "2.0",
  "id": 4,
  "method": "tools/call",
  "params": {
    "name": "getRecentErrors",
    "arguments": {
      "service_name": "payment-service",
      "limit": 5
    }
  }
}
```

**Response** — ordered from most recent:
1. `ERROR` — DB connection pool exhausted (max 100 reached)
2. `CRITICAL` — Transaction timeout after 30 s, DB unresponsive
3. `WARN` — Connection pool at 90%, approaching limit

Root cause is clear: the database is overloaded and the connection pool is full.

---

### Request 5 — `tools/call → createTicket`

With the diagnosis confirmed, create a formal incident ticket.

**Request:**

```json
{
  "jsonrpc": "2.0",
  "id": 5,
  "method": "tools/call",
  "params": {
    "name": "createTicket",
    "arguments": {
      "title": "payment-service DB connection pool exhausted",
      "severity": "critical",
      "description": "DB connection pool reached max (100). Transactions timing out after 30s. notification-service also down with OOM crash. Immediate DB scaling and connection pool tuning required."
    }
  }
}
```

**Response:**

```json
{
  "success": true,
  "message": "Ticket INC-1001 created successfully",
  "ticket": {
    "ticket_id": "INC-1001",
    "severity": "critical",
    "status": "open",
    "assigned_to": "on-call-team"
  }
}
```

---

### Request 6 — `prompts/list`

Discover the structured workflow templates the server exposes.

**Request:**

```json
{
  "jsonrpc": "2.0",
  "id": 6,
  "method": "prompts/list",
  "params": {}
}
```

**Response:**

```json
{
  "jsonrpc": "2.0",
  "id": 6,
  "result": {
    "prompts": [
      {
        "name": "incidentAnalysisTemplate",
        "description": "A structured template for analyzing IT incidents. Guides the LLM through a systematic diagnosis workflow."
      }
    ]
  }
}
```

A **prompt** in MCP is not the user's message — it is a **workflow template the server publishes**. An LLM agent can fetch `incidentAnalysisTemplate` at runtime and follow its prescribed steps:

```
ASSESS → INVESTIGATE → CORRELATE → DIAGNOSE → RECOMMEND → ACT
```

Different servers publish different workflows. The agent adapts automatically without any code change on the client side.

---

### The full incident workflow — what just happened

```
Request 1: initialize
├── Client: "Hi server, what can you do?"
└── Server: "I have tools, prompts, and resources"

Request 2: tools/list
├── Client: "Show me your tools"
└── Server: "getSystemStatus, getRecentErrors, createTicket"

Request 3: tools/call → getSystemStatus
├── Client: "What's the current system health?"
└── Server: "payment-service DEGRADED, notification-service DOWN"

Request 4: tools/call → getRecentErrors
├── Client: "Show me payment-service errors"
└── Server: "DB connection pool exhausted, transaction timeouts"

Request 5: tools/call → createTicket
├── Client: "Create critical ticket with diagnosis"
└── Server: "INC-1001 created, assigned to on-call-team" ✅

Request 6: prompts/list
├── Client: "What workflow templates do you have?"
└── Server: "incidentAnalysisTemplate (ASSESS→INVESTIGATE→...→ACT)"
```

### Phase 2 vs Phase 3 — the key difference

**Phase 2 (done):** You are the brain. You read each response, decided what to call next, and wrote the ticket description manually.

**Phase 3 (next):** The LLM is the brain. Given a single user message — `"There's an alert, what's going on?"` — it will call the same sequence of tools automatically, reason over each response, and produce the ticket without any manual input.

The LLM fills in the same arguments you typed, but generates them from reasoning over previous tool outputs. The server does not change at all between Phase 2 and Phase 3.

> **Phase 2 is complete.** All 6 JSON-RPC methods verified, full incident lifecycle exercised end-to-end.