# observability-mcp-gateway

A **remote** Model Context Protocol (MCP) server for a synthetic observability
dataset - logs, latency/error-rate metrics, and alerts for five fictional
microservices - plus a small client that drives it over the network. It's a
learning project built to get hands-on with the "remote MCP server" pattern:
HTTP + Server-Sent-Events instead of the more commonly-documented local
stdio server.

Everything here is fake data generated by a seeded script (`src/data/generator.ts`).
No real services, no real incidents. The point is the MCP plumbing, not the data.

## Why remote (HTTP) instead of stdio?

Most MCP tutorials show the **stdio transport**: your MCP client (an IDE
plugin, a desktop AI assistant app, an agent framework) spawns your server
as a local child process and talks to it over stdin/stdout. That's simple
and it's the right choice when:

- the server only ever needs to run on the same machine as the client,
- you're fine with the client managing the server's lifecycle (start/stop/crash-restart),
- there's exactly one client talking to one server instance at a time.

A **remote MCP server** (what this repo builds) runs independently, reachable
over HTTP, and clients connect to it like any other web API. That's the right
choice when:

- multiple clients/users need to share one running server (and its state -
  here, one SQLite-backed dataset everyone queries),
- the server needs to live somewhere other than the client's machine (a
  teammate's laptop can't spawn a process on your server),
- you want normal ops tooling to apply: a process manager, a reverse proxy,
  horizontal scaling, logs/metrics on the server itself, independent deploys.

The trade-off is that a remote server has to solve problems stdio gets for
free: network transport framing, reconnection, and - the one stdio servers
never have to think about - **access control**. Anyone who can reach the
port can call your tools, so this repo includes a bearer-token gate (see
[Auth](#auth-approach) below).

The SDK's **StreamableHTTP** transport is the current standard way to do
this: a single `/mcp` endpoint that accepts JSON-RPC requests over HTTP POST
and can upgrade individual responses to Server-Sent-Events for
streaming/long-running calls. It replaced the older plain-SSE transport from
earlier versions of the spec.

This server runs StreamableHTTP in **stateless mode** - no session ID, no
long-lived connection, a fresh `McpServer` per request. That's a deliberate
simplification: every tool here is a quick read against SQLite, so there's
no need for server-initiated push notifications or resumable streams, and
stateless mode means no sticky-session requirements if you ever put a load
balancer in front of it. A chattier server (one that pushes progress updates
or long-running job notifications) would want the stateful mode instead -
one `McpServer` per client session, tracked by a session ID header.

## Architecture

```
  ┌───────────────┐   POST /mcp (JSON-RPC)    ┌──────────────────────────────┐
  │   MCP client   │   Authorization: Bearer   │   observability-mcp-gateway  │
  │ (src/client,   │ ─────────────────────────▶│                              │
  │  or any MCP    │                            │  express                    │
  │  host app)     │◀───────────────────────── │   └─ bearerAuth middleware   │
  └───────────────┘   JSON-RPC result / SSE    │        └─ StreamableHTTP     │
                                                 │             Transport       │
                                                 │                 └─ McpServer│
                                                 │                      │      │
                                                 │           ┌──────────┴────┐ │
                                                 │           │               │ │
                                                 │        tools          resource│
                                                 │   (search_logs, etc.) (health)│
                                                 │           │               │ │
                                                 │           └──────┬────────┘ │
                                                 │                  ▼          │
                                                 │        ObservabilityStore   │
                                                 │         (SQLite, in-memory) │
                                                 │                  ▲          │
                                                 │        seeded once at boot  │
                                                 │         generateDataset()   │
                                                 └──────────────────────────────┘
```

Request flow for one tool call:

1. Client sends `POST /mcp` with a JSON-RPC `tools/call` body and an
   `Authorization: Bearer <token>` header.
2. `bearerAuth` middleware checks the header before anything else runs.
3. A fresh `StreamableHTTPServerTransport` + `McpServer` pair handles the
   request (stateless mode - see above).
4. The tool handler calls into `ObservabilityStore`, which runs an indexed
   SQL query against the in-memory SQLite database seeded at server startup.
5. The result comes back as both `structuredContent` (typed, per the tool's
   `outputSchema`) and a human-readable `content` text block.

## Repo layout

```
src/
  data/
    types.ts       domain types (LogEntry, MetricPoint, AlertRecord, ...)
    rng.ts          seeded PRNG (mulberry32) for reproducible fake data
    generator.ts    builds the synthetic dataset (logs/metrics/alerts)
    store.ts        SQLite-backed query layer the tools call into
  server/
    auth.ts         bearer-token middleware
    mcpServer.ts     builds the McpServer and registers tools + resource
    httpServer.ts    express app: health check, auth, /mcp route
    index.ts         process entrypoint (reads env, starts listening)
    tools/           the 5 MCP tools + their Zod schemas
    resources/       the live service-health resource
  client/
    index.ts         demo MCP client over StreamableHTTP
tests/
  store.test.ts       unit tests for the query layer
  auth.test.ts        unit tests for the auth middleware
  integration.test.ts  starts the real HTTP server + real SDK client
```

## Setup

Requires Node 20+.

```bash
npm install
cp .env.example .env     # then edit MCP_GATEWAY_TOKEN to something real
```

## Running it

Start the server (reads `MCP_GATEWAY_TOKEN`, `HOST`, `PORT` from the
environment - `npm run dev` loads `.env` via your shell, or export them
inline as shown):

```bash
MCP_GATEWAY_TOKEN=dev-secret npm run dev
```

(PowerShell: `$env:MCP_GATEWAY_TOKEN = "dev-secret"; npm run dev`)

```
observability-mcp-gateway listening on http://127.0.0.1:8787/mcp
health check: http://127.0.0.1:8787/healthz
auth: bearer token required (MCP_GATEWAY_TOKEN)
```

In another terminal, run the demo client against it:

```bash
MCP_GATEWAY_TOKEN=dev-secret npm run client
```

(PowerShell: `$env:MCP_GATEWAY_TOKEN = "dev-secret"; npm run client`)

Other scripts:

```bash
npm run seed        # print a summary of the generated dataset, no server needed
npm run build        # tsc -> dist/
npm start             # run the compiled server (dist/server/index.js)
npm test              # vitest: unit + integration tests
npm run lint          # eslint
npm run typecheck     # tsc --noEmit
```

## Example tool calls

Output below is a lightly trimmed real run of `npm run client` against a
freshly-seeded server (the dataset is deterministic per server start, so
service names/alert text will match; exact timestamps/ids will differ).

**`search_logs`** - errors mentioning "timed out" on `payments-gateway`:

```json
{
  "logs": [
    {
      "id": 2879,
      "timestamp": "2026-10-01T13:26:06.344Z",
      "service": "payments-gateway",
      "severity": "error",
      "message": "downstream request timed out",
      "traceId": "47d3695d616ca81d"
    }
  ],
  "matchCount": 5
}
```

**`get_service_health`** - `payments-gateway` (seeded with an ongoing incident):

```json
{
  "service": "payments-gateway",
  "status": "down",
  "latencyP95Ms": 652,
  "errorRatePct": 12.99,
  "activeAlertCount": 1,
  "lastUpdated": "2026-10-01T13:28:08.567Z"
}
```

**`list_active_alerts`**:

```json
{
  "alerts": [
    {
      "id": 1,
      "service": "payments-gateway",
      "severity": "critical",
      "status": "firing",
      "title": "Elevated error rate on payments-gateway",
      "description": "Error rate on payments-gateway has exceeded the 5% threshold for the last 15+ minutes, correlated with a spike in upstream processor timeouts.",
      "startedAt": "2026-10-01T12:46:08.567Z",
      "resolvedAt": null
    }
  ],
  "count": 1
}
```

**`get_latency_percentiles`** - `checkout-api`, trailing 60 minutes:

```json
{ "service": "checkout-api", "windowMinutes": 60, "sampleCount": 60, "p50": 118, "p95": 149, "p99": 155 }
```

**`query_recent_errors`** - `notifications-worker`:

```json
{
  "errors": [
    { "id": 5800, "timestamp": "2026-10-01T13:20:09.032Z", "service": "notifications-worker", "severity": "error", "message": "unhandled exception while processing request", "traceId": "ff0779e5ef9ead10" }
  ],
  "count": 5
}
```

**resource read** - `observability://service-health/snapshot` (abridged):

```json
{
  "generatedAt": "2026-10-01T13:28:11.911Z",
  "services": [
    { "service": "checkout-api", "status": "healthy", "latencyP95Ms": 149, "errorRatePct": 0.55, "activeAlertCount": 0, "lastUpdated": "..." },
    { "service": "payments-gateway", "status": "down", "latencyP95Ms": 652, "errorRatePct": 12.99, "activeAlertCount": 1, "lastUpdated": "..." }
  ]
}
```

## Auth approach

The `/mcp` route is gated by `bearerAuth` (`src/server/auth.ts`): every
request must carry `Authorization: Bearer <MCP_GATEWAY_TOKEN>`, checked with
a constant-time comparison (`crypto.timingSafeEqual`) so token-guessing can't
be sped up by timing the response. No token configured means the server
refuses to start at all - there's no "accidentally open" default.

This is intentionally the simplest thing that is still *correct* access
control, not a production-grade auth system. It's here to demonstrate the
point that a remote MCP server is a network service and needs one - a local
stdio server gets access control for free because only the process that
spawned it can talk to it; a remote one does not.

To harden this for a real deployment:

- **Swap in OAuth 2.1.** The MCP auth spec (2025-03-26+) expects a remote
  server to act as an OAuth **resource server**: serve
  `/.well-known/oauth-protected-resource`, validate access tokens (JWT or
  opaque, verified against your IdP), and return `WWW-Authenticate` challenges
  per RFC 9728 instead of a bare 401. The SDK ships this as
  `requireBearerAuth` (`@modelcontextprotocol/sdk/server/auth/...`) plus
  `mcpAuthRouter` - the shape of the change is replacing `bearerAuth(...)`
  with that middleware and pointing it at a real `OAuthTokenVerifier`.
- **Scope tokens per capability.** Right now one token can call every tool.
  A real deployment would mint tokens with scopes like `observability:read`
  and check them per tool/resource instead of all-or-nothing.
- **Rotate and expire tokens.** A static long-lived secret in `.env` is fine
  for a side project; it should not survive contact with real users.
- **Rate-limit and log auth failures** separately from normal traffic, so a
  credential-stuffing attempt is visible before it succeeds.
- **Put TLS in front of it.** This server assumes it sits behind a
  TLS-terminating proxy; bearer tokens over plain HTTP are only as safe as
  the network they cross.

## Lessons learned

A few things that weren't obvious going in:

- **Stateless StreamableHTTP is a real, supported mode**, not a hack - the
  SDK ships an official example for it. For a server whose tools are all
  quick reads, it removes an entire class of ops problems (session affinity,
  reconnection logic) that the stateful mode has to solve.
- **`outputSchema` + `structuredContent` is worth the extra typing.** It's
  tempting to just stuff a JSON string into a text block and call it done,
  but declaring the output shape means any MCP client that understands
  structured content gets typed data for free, and the schema doubles as
  documentation of exactly what a tool returns.
- **"Synthetic data" is easy to make too clean.** The first pass at the
  generator produced tidy, uniform log lines and nobody would believe it was
  real telemetry. Adding deliberate mess - inconsistent casing, stray
  whitespace, occasional missing trace IDs - made `search_logs` demos feel
  like they were querying an actual system instead of a fixture.
- **A remote server's biggest conceptual shift from stdio isn't the
  transport, it's ownership.** A stdio server's lifecycle is the client's
  problem. A remote server has to manage its own startup, shutdown, and
  access control, because nobody is holding its hand.