# FDE Assessment — MCP servers, gateways, guardrails and model routing

Four runnable projects, one Python package layout, one test suite.
**219 tests, all passing.** This README covers the design of each task; the
reasoning behind individual decisions is commented at the point in the code
where it applies.

```
fde-assessment/
├── task1_mcp_server/        MCP server, strict validation, stdio isolation
├── task2_mcp_gateway/       MCP security gateway (tool-level authorization)
├── task3_stream_guardrail/  LLM gateway with streaming PII redaction
├── task4_router/            rate limiter + model failover router (SQLite)
├── requirements.txt
├── Makefile                 setup / test / run targets
└── pytest.ini
```

## Setup

```bash
make setup          # python3 -m venv .venv && pip install -r requirements.txt
make test           # 219 tests, ~12s
```

Python 3.11+ (task 4 uses `asyncio.timeout`). Tested on 3.13.

---

## The one idea behind all four tasks

An LLM gateway is a **policy layer on a protocol you do not control**. Each task
is a different place that policy has to live, and each has one failure mode that
separates working code from code that merely passes a demo:

| Task | The policy | The failure mode that matters |
|---|---|---|
| 1 | schema validation at the tool boundary | a stray `print()` corrupts the JSON-RPC stream |
| 2 | who may call which tool | authorization that reads the URL instead of the JSON-RPC body |
| 3 | no PII leaves the gateway | PII split across two stream chunks; or buffering the whole response to catch it |
| 4 | fair use + resilience | retrying an error the backup will also return; leaking upstream internals |

---

## Task 1 — MCP server with strict validation and stdio isolation

**Files:** [`task1_mcp_server/server.py`](task1_mcp_server/server.py) ·
[`test_task1.py`](task1_mcp_server/test_task1.py) · 22 tests

Two tools over the official MCP Python SDK (`mcp` 2.x, `mcp.server.lowlevel.Server`):

- `get_customer_record(customer_id)` — `CUST-XXXXX` format enforced by regex
- `trigger_refund(customer_id, amount, reason)` — amount > 0 and ≤ 10000, reason ≥ 10 non-whitespace chars

```bash
make task1                            # speaks JSON-RPC on stdin/stdout
make test-1
```

### The three graded points

**1. STDIO isolation.** stdout is the wire. One `print()` and the client
disconnects with a parse error. Two layers of defence:

- `configure_logging()` pins logging to an explicit stderr handler.
- The SDK's `stdio_server()` duplicates the real fd 1 for the transport and
  repoints fd 1 at stderr while serving — so a stray `print()` **anywhere**,
  including in a dependency or a child process, lands on stderr.

The test suite proves it rather than asserting it: run the server with
`LEAK_DEMO=1` and the refund handler prints junk on purpose. The test then
asserts every line of stdout still parses as JSON-RPC 2.0, and that the junk
turned up on stderr.

**2. Protocol compliance — the distinction most implementations miss:**

| Situation | Response | Why |
|---|---|---|
| `amount: -5`, bad `customer_id`, unknown field | JSON-RPC **error −32602** | the request was not a valid thing to ask |
| unknown tool name | JSON-RPC **error −32601** | not in our advertised surface |
| customer doesn't exist, refund exceeds balance | **result** with `isError: true` | the call was fine; the *world* said no — the model is meant to read this and adapt |

Collapsing these into one channel is what makes agents loop: a model that gets a
protocol error for "customer not found" retries the same call forever.

**3. Validation.** Pydantic with `extra="forbid"` (a `{"ammount": 10}` typo
fails loudly instead of defaulting) and `strict=True` (the string `"50"` is not
silently coerced to `50.0`). Plus the edge cases a regex alone misses: `NaN`/
`inf`, sub-cent precision like `10.005`, and a `reason` of ten spaces.
`tools/list` publishes the *same* Pydantic schema that validates the call, so
the advertised contract and the enforced contract cannot drift apart.

---

## Task 2 — MCP security gateway

**Files:** [`gateway.py`](task2_mcp_gateway/gateway.py) ·
[`downstream_mcp_server.py`](task2_mcp_gateway/downstream_mcp_server.py) ·
[`test_task2.py`](task2_mcp_gateway/test_task2.py) · 19 tests

```
agent ──▶ gateway :9000 ──▶ downstream MCP :9001
             ├─ Bearer token → principal + role
             ├─ tools/list  → forwarded transparently
             └─ tools/call  → admin_* requires role=admin, else −32001
```

```bash
make task2-downstream        # terminal 1
make task2-gateway           # terminal 2

curl -s localhost:9000/mcp -H 'Authorization: Bearer viewer-token-xyz' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"admin_reset_key"}}'
# {"jsonrpc":"2.0","id":1,"error":{"code":-32001,"message":"Unauthorized Tool Call",
#  "data":{"tool":"admin_reset_key","required_role":"admin","your_role":"viewer"}}}
```

### Why this cannot be done with a normal API gateway

Every MCP call is a `POST` to the same URL. Path- and verb-based rules therefore
authorize *nothing* — `tools/list` and `admin_delete_tenant` are the same HTTP
request on the wire. The decision has to read `params.name` out of the JSON-RPC
body, which is what `authorize()` does.

### Decisions worth calling out

- **A denied call never reaches the downstream server.** The mock records every
  tool it executes and the tests assert that log stays empty — "denied, but
  forwarded anyway" is a real and common bug.
- **−32001 is returned with HTTP 200.** MCP clients parse the JSON-RPC body; a
  bare HTTP 403 with an HTML error page is what breaks agents in the field.
  *Authentication* failures are different — those get HTTP 401 **and** a
  JSON-RPC −32000 body, so both layers can understand them.
- **Batches are authorized element by element.** A batch mixing one allowed and
  one denied call forwards only the allowed element and merges the denial back
  in by id, preserving request order. Ignoring batches is the classic way an
  authz proxy gets bypassed.
- **Case-insensitive prefix check** — `Admin_Reset_Key` does not slip past.
- **The client's token is not forwarded.** The gateway re-authenticates to the
  backend with its own service credential, so a leaked user token is useless
  against the downstream directly.
- **`HIDE_ADMIN_TOOLS=1`** additionally strips `admin_*` from `tools/list` for
  non-admins, so a viewer's model never learns those tools exist. Off by
  default because the brief asks for `tools/list` to be transparent.

---

## Task 3 — LLM gateway with a streaming PII guardrail

**Files:** [`redactor.py`](task3_stream_guardrail/redactor.py) ·
[`gateway.py`](task3_stream_guardrail/gateway.py) ·
[`mock_llm_provider.py`](task3_stream_guardrail/mock_llm_provider.py) ·
[`test_task3.py`](task3_stream_guardrail/test_task3.py) · 151 tests

```bash
make task3-provider          # terminal 1
make task3-gateway           # terminal 2

curl -N localhost:9100/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"m","stream":true,"messages":[{"role":"user","content":"hi"}]}'
```

Live output (TTFT measured at **26 ms**, upstream streams for ~150 ms):

```
Sure! Here is the account owner: [REDACTED_EMAIL]. Their SSN is [REDACTED_SSN]
and the card on file is [REDACTED_CARD]. The internal key is [REDACTED_SECRET].
Anything else?
: guardrail redactions={'email': 1, 'ssn': 1, 'credit_card': 1, 'api_key': 1}
```

The mock provider deliberately cuts every secret across SSE frames
(`"4111 1111"` + `" 1111 1111"`), because that is the case a per-chunk regex
silently misses.

### The algorithm: hold-back

Per chunk:

1. Redact every **complete** match in the buffer.
2. Find the earliest position from which a match could still be **in progress**
   — the *hold point* — using `regex`'s `partial=True`.
3. Emit everything before the hold point; keep the rest.

Text before the hold point is provably safe: if a match could start there and
extend into future input, `regex` would have reported a partial match at that
position. Two things count as "in progress": a genuinely partial match
(`4111 11`), and a **complete match that ends at the buffer's end** — valid
right now, but the next chunk could lengthen it into something different.

> That second case was a real bug caught by the exhaustive split test: in
> `sk-abcdef012345678` the last 9 digits form a complete SSN candidate while an
> API-key match is still in progress from offset 0. Cutting at the SSN's start
> would have leaked the key. The fix is in `_hold_point()`, and the comment
> there explains it.

**Properties:**

- **Memory is O(window), not O(response)** — capped at `max_match_len` (64
  chars). A test streams ~450 KB through and asserts the buffer never exceeds
  the bound. Unbounded buffering in a gateway is a DoS vector, not just a
  latency problem.
- **TTFT stays low** — ordinary prose holds back ~0 characters, so the first
  token leaves the gateway essentially as fast as it arrived. Only text that
  could still be PII waits.
- **Correctness is tested exhaustively** — the same PII string is fed through
  *every possible two-way split* (147 parametrized cases), one character at a
  time, and 200 random chunkings. All must produce byte-identical output.

### Other decisions

- **`regex` over stdlib `re`.** `re` can only answer "did this match?", never
  "could this still become a match?". Without partial matching you must always
  hold back the full window, adding latency to every stream.
- **Validators kill false positives.** A card candidate must pass **Luhn**; an
  SSN must not be in a never-issued range. Redacting a customer's order number
  as a credit card is its own kind of incident.
- **One redactor per choice index.** With `n > 1` the provider interleaves
  completions on one stream; a shared redactor would splice choice 0's
  held-back tail onto choice 1's text.
- **Streaming is preserved end to end** — `client.stream()` + `aiter_bytes()`,
  an incremental `SSEDecoder` that holds at most one partial event, and
  `X-Accel-Buffering: no` so nginx doesn't undo all of it in production.
- **`flush()` at end of stream** — a secret in the final fragment is only
  releasable then. There's a test for exactly that truncation bug.
- The TTFT test runs against **real sockets** (uvicorn on ephemeral ports),
  because httpx's in-process ASGI transport buffers the whole body and cannot
  measure time-to-first-token at all.

---

## Task 4 — rate limiting and model failover

**Files:** [`rate_limiter.py`](task4_router/rate_limiter.py) ·
[`router.py`](task4_router/router.py) · [`db.py`](task4_router/db.py) ·
[`errors.py`](task4_router/errors.py) · [`app.py`](task4_router/app.py) ·
[`mock_providers.py`](task4_router/mock_providers.py) ·
[`test_task4.py`](task4_router/test_task4.py) · 27 tests

```bash
make task4-providers         # terminal 1
make task4-gateway           # terminal 2

curl -s localhost:9200/v1/chat/completions -i \
  -H 'Authorization: Bearer sk-tenant-acme' -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"hello"}],"max_tokens":100}'

# flip the primary's behaviour to watch failover live:
curl -X POST 'localhost:9201/_set?provider=primary&behaviour=429'   # or 500 / slow / 400
```

Live run against the real servers:

```
primary=ok    -> 200 provider=primary   attempts=primary:ok                      0.01s
primary=429   -> 200 provider=backup    attempts=primary:rate_limited,backup:ok  0.00s
primary=slow  -> 200 provider=backup    attempts=primary:timeout,backup:ok       3.01s   ← deadline honoured
primary=400   -> 400 gateway.upstream_rejected_request                           0.00s   ← no pointless retry
both down     -> 502 gateway.all_providers_failed
```

### Rate limiter: sliding window on on-disk SQLite

A fixed 60-second bucket lets a tenant spend 50k tokens at 11:59:59 and another
50k at 12:00:00 — the exact burst the limit exists to prevent. This stores one
row per request and sums the rows inside the trailing 60 seconds, so the window
moves with the clock. Expired rows are evicted on the write path, so the table
stays proportional to live traffic with no background job.

**Reserve → settle → release.** Token cost isn't known until the response
arrives, but the decision must be made before the call:

1. `reserve()` — estimate (prompt + `max_tokens`), check and insert atomically.
   Deliberately conservative.
2. `settle()` — the response's real usage replaces the estimate. A test proves
   a `max_tokens=4000` request settles at ~43 tokens: a tenant is not billed for
   their own generosity in `max_tokens`.
3. `release()` — the call failed, so the reservation is dropped. **Our outage
   does not consume the tenant's budget** (also tested).

**The race that matters.** Read-then-write is not atomic: two concurrent
requests both read "40k used of 50k" and both proceed. `BEGIN IMMEDIATE` takes
the write lock *before* the SELECT. The test fires 50 concurrent reservations of
1000 tokens against a 10000 limit and asserts **exactly 10** are allowed.

**Why on-disk and not a dict.** In-memory state resets every deploy and is not
shared between workers — both are how a tenant ends up with 2× their limit. WAL,
`busy_timeout` and `synchronous=NORMAL` are what make SQLite behave under
concurrency; every call is wrapped in `asyncio.to_thread` so a disk stall never
blocks the event loop. A test restarts the limiter against the same file and
asserts the quota survived.

**A 60k-token request against a 50k limit returns 413, not 429.** A 429 would be
a lie — no amount of waiting makes it fit. `Retry-After` is also computed
honestly: the limiter walks the window oldest-first to find when enough tokens
actually expire, instead of returning a flat 60 and creating a thundering herd
at the top of every minute.

### Failover policy

| Upstream result | Fail over? | Why |
|---|---|---|
| 429 | yes | the backup has its own quota |
| timeout > 3000 ms | yes | the primary is degraded |
| 5xx, connection error, unparseable 200 | yes | the primary is down |
| 400 / 422 | **no** | malformed request — the backup rejects it identically |
| 401 / 403 | **no** | *our* credential is wrong |

Retrying an unretryable error isn't resilience; it's a second way to be slow
before returning the same failure.

**The timeout is a total deadline.** `asyncio.timeout(3.0)` wraps the whole
attempt — DNS, connect, TLS, headers, body. Per-phase HTTP timeouts miss the
failure that actually hurts: a provider dribbling bytes just fast enough to keep
resetting the read timer while blowing the SLA. Cancelling also tears down the
socket, so a timed-out attempt can't hold a pooled connection.

**Circuit breaker.** After N consecutive failures the primary is skipped
entirely for a cooldown, then half-opens for one probe. Without it, every
request pays the full 3 s timeout while the primary is down and a provider
outage becomes a gateway outage.

### Error sanitization

One envelope for every failure:

```json
{"error": {"type": "upstream_error", "code": "gateway.all_providers_failed",
           "message": "No model provider could serve this request. Please retry shortly.",
           "request_id": "f016f46bd42f4fd3afbb940332b46b18"}}
```

The mock providers return deliberately leaky bodies —
`db-3.us-east-1.internal`, `quota-svc-7.internal`, `org-4417` — and parametrized
tests assert none of it appears in any client response. The detail is logged
against the same `request_id` the client received: that id is the join key, so
support can answer "what happened to request f016…" without the gateway having
leaked anything. A catch-all handler ensures not even a bug in the gateway
escapes un-sanitized.

---

## Test summary

```
$ make test
219 passed in 11.33s

task1_mcp_server      22   stdio purity under a deliberate print(), error-code mapping, 13 malformed-input cases
task2_mcp_gateway     19   authz, batches, notifications, auth failures, sanitization
task3_stream_guardrail 151  every two-way split of a PII string, char-by-char, 200 random chunkings,
                            memory bound over 450 KB, real-socket TTFT
task4_router          27   window eviction, 50-way concurrency race, restart persistence,
                            every failover branch, circuit breaker, leak assertions
```

## What I'd add next, given more time

- **Task 1:** a `refund_id` idempotency key so a retried tool call can't double-refund.
- **Task 2:** JWT/JWKS instead of the static token table, and per-tenant tool
  allowlists loaded from the control plane rather than a hardcoded `admin_` prefix.
- **Task 3:** redact the *request* as well as the response (prompts leak PII too),
  and add a semantic classifier behind the regex layer for the things patterns
  can't catch (names, addresses).
- **Task 4:** Redis or a shared store once the gateway runs on more than one
  node — SQLite is per-node state, which is correct for a single VM and wrong
  for a fleet. Streaming support in the router, with failover before the first
  token is committed to the client.