FDE Assessment — MCP servers, gateways, guardrails and model routing

Four runnable projects, one Python package layout, one test suite. 219 tests, all passing. This README covers the design of each task; the reasoning behind individual decisions is commented at the point in the code where it applies.

fde-assessment/
├── task1_mcp_server/        MCP server, strict validation, stdio isolation
├── task2_mcp_gateway/       MCP security gateway (tool-level authorization)
├── task3_stream_guardrail/  LLM gateway with streaming PII redaction
├── task4_router/            rate limiter + model failover router (SQLite)
├── requirements.txt
├── Makefile                 setup / test / run targets
└── pytest.ini

Setup

make setup          # python3 -m venv .venv && pip install -r requirements.txt
make test           # 219 tests, ~12s

Python 3.11+ (task 4 uses asyncio.timeout). Tested on 3.13.

Related MCP server: SentinelMCP

The one idea behind all four tasks

An LLM gateway is a policy layer on a protocol you do not control. Each task is a different place that policy has to live, and each has one failure mode that separates working code from code that merely passes a demo:

Task 1 — MCP server with strict validation and stdio isolation

Files: task1_mcp_server/server.py · test_task1.py · 22 tests

Two tools over the official MCP Python SDK (mcp 2.x, mcp.server.lowlevel.Server):

• get_customer_record(customer_id) — CUST-XXXXX format enforced by regex

• trigger_refund(customer_id, amount, reason) — amount > 0 and ≤ 10000, reason ≥ 10 non-whitespace chars

make task1                            # speaks JSON-RPC on stdin/stdout
make test-1

The three graded points

1. STDIO isolation. stdout is the wire. One print() and the client disconnects with a parse error. Two layers of defence:

• configure_logging() pins logging to an explicit stderr handler.

• The SDK's stdio_server() duplicates the real fd 1 for the transport and repoints fd 1 at stderr while serving — so a stray print() anywhere, including in a dependency or a child process, lands on stderr.

The test suite proves it rather than asserting it: run the server with LEAK_DEMO=1 and the refund handler prints junk on purpose. The test then asserts every line of stdout still parses as JSON-RPC 2.0, and that the junk turned up on stderr.

2. Protocol compliance — the distinction most implementations miss:

Collapsing these into one channel is what makes agents loop: a model that gets a protocol error for "customer not found" retries the same call forever.

3. Validation. Pydantic with extra="forbid" (a {"ammount": 10} typo fails loudly instead of defaulting) and strict=True (the string "50" is not silently coerced to 50.0). Plus the edge cases a regex alone misses: NaN/ inf, sub-cent precision like 10.005, and a reason of ten spaces. tools/list publishes the same Pydantic schema that validates the call, so the advertised contract and the enforced contract cannot drift apart.

Task 2 — MCP security gateway

Files: gateway.py · downstream_mcp_server.py · test_task2.py · 19 tests

agent ──▶ gateway :9000 ──▶ downstream MCP :9001
             ├─ Bearer token → principal + role
             ├─ tools/list  → forwarded transparently
             └─ tools/call  → admin_* requires role=admin, else −32001

make task2-downstream        # terminal 1
make task2-gateway           # terminal 2

curl -s localhost:9000/mcp -H 'Authorization: Bearer viewer-token-xyz' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"admin_reset_key"}}'
# {"jsonrpc":"2.0","id":1,"error":{"code":-32001,"message":"Unauthorized Tool Call",
#  "data":{"tool":"admin_reset_key","required_role":"admin","your_role":"viewer"}}}

Why this cannot be done with a normal API gateway

Every MCP call is a POST to the same URL. Path- and verb-based rules therefore authorize nothing — tools/list and admin_delete_tenant are the same HTTP request on the wire. The decision has to read params.name out of the JSON-RPC body, which is what authorize() does.

Decisions worth calling out

• A denied call never reaches the downstream server. The mock records every tool it executes and the tests assert that log stays empty — "denied, but forwarded anyway" is a real and common bug.

• −32001 is returned with HTTP 200. MCP clients parse the JSON-RPC body; a bare HTTP 403 with an HTML error page is what breaks agents in the field. Authentication failures are different — those get HTTP 401 and a JSON-RPC −32000 body, so both layers can understand them.

• Batches are authorized element by element. A batch mixing one allowed and one denied call forwards only the allowed element and merges the denial back in by id, preserving request order. Ignoring batches is the classic way an authz proxy gets bypassed.

• Case-insensitive prefix check — Admin_Reset_Key does not slip past.

• The client's token is not forwarded. The gateway re-authenticates to the backend with its own service credential, so a leaked user token is useless against the downstream directly.

• HIDE_ADMIN_TOOLS=1 additionally strips admin_* from tools/list for non-admins, so a viewer's model never learns those tools exist. Off by default because the brief asks for tools/list to be transparent.

Task 3 — LLM gateway with a streaming PII guardrail

Files: redactor.py · gateway.py · mock_llm_provider.py · test_task3.py · 151 tests

make task3-provider          # terminal 1
make task3-gateway           # terminal 2

curl -N localhost:9100/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"m","stream":true,"messages":[{"role":"user","content":"hi"}]}'

Live output (TTFT measured at 26 ms, upstream streams for ~150 ms):

Sure! Here is the account owner: [REDACTED_EMAIL]. Their SSN is [REDACTED_SSN]
and the card on file is [REDACTED_CARD]. The internal key is [REDACTED_SECRET].
Anything else?
: guardrail redactions={'email': 1, 'ssn': 1, 'credit_card': 1, 'api_key': 1}

The mock provider deliberately cuts every secret across SSE frames ("4111 1111" + " 1111 1111"), because that is the case a per-chunk regex silently misses.

The algorithm: hold-back

Per chunk:

1. Redact every complete match in the buffer.

2. Find the earliest position from which a match could still be in progress — the hold point — using regex's partial=True.

3. Emit everything before the hold point; keep the rest.

Text before the hold point is provably safe: if a match could start there and extend into future input, regex would have reported a partial match at that position. Two things count as "in progress": a genuinely partial match (4111 11), and a complete match that ends at the buffer's end — valid right now, but the next chunk could lengthen it into something different.

That second case was a real bug caught by the exhaustive split test: in sk-abcdef012345678 the last 9 digits form a complete SSN candidate while an API-key match is still in progress from offset 0. Cutting at the SSN's start would have leaked the key. The fix is in _hold_point(), and the comment there explains it.

Properties:

• Memory is O(window), not O(response) — capped at max_match_len (64 chars). A test streams ~450 KB through and asserts the buffer never exceeds the bound. Unbounded buffering in a gateway is a DoS vector, not just a latency problem.

• TTFT stays low — ordinary prose holds back ~0 characters, so the first token leaves the gateway essentially as fast as it arrived. Only text that could still be PII waits.

• Correctness is tested exhaustively — the same PII string is fed through every possible two-way split (147 parametrized cases), one character at a time, and 200 random chunkings. All must produce byte-identical output.

Other decisions

• regex over stdlib re. re can only answer "did this match?", never "could this still become a match?". Without partial matching you must always hold back the full window, adding latency to every stream.

• Validators kill false positives. A card candidate must pass Luhn; an SSN must not be in a never-issued range. Redacting a customer's order number as a credit card is its own kind of incident.

• One redactor per choice index. With n > 1 the provider interleaves completions on one stream; a shared redactor would splice choice 0's held-back tail onto choice 1's text.

• Streaming is preserved end to end — client.stream() + aiter_bytes(), an incremental SSEDecoder that holds at most one partial event, and X-Accel-Buffering: no so nginx doesn't undo all of it in production.

• flush() at end of stream — a secret in the final fragment is only releasable then. There's a test for exactly that truncation bug.

• The TTFT test runs against real sockets (uvicorn on ephemeral ports), because httpx's in-process ASGI transport buffers the whole body and cannot measure time-to-first-token at all.

Task 4 — rate limiting and model failover

Files: rate_limiter.py · router.py · db.py · errors.py · app.py · mock_providers.py · test_task4.py · 27 tests

make task4-providers         # terminal 1
make task4-gateway           # terminal 2

curl -s localhost:9200/v1/chat/completions -i \
  -H 'Authorization: Bearer sk-tenant-acme' -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"hello"}],"max_tokens":100}'

# flip the primary's behaviour to watch failover live:
curl -X POST 'localhost:9201/_set?provider=primary&behaviour=429'   # or 500 / slow / 400

Live run against the real servers:

primary=ok    -> 200 provider=primary   attempts=primary:ok                      0.01s
primary=429   -> 200 provider=backup    attempts=primary:rate_limited,backup:ok  0.00s
primary=slow  -> 200 provider=backup    attempts=primary:timeout,backup:ok       3.01s   ← deadline honoured
primary=400   -> 400 gateway.upstream_rejected_request                           0.00s   ← no pointless retry
both down     -> 502 gateway.all_providers_failed

Rate limiter: sliding window on on-disk SQLite

A fixed 60-second bucket lets a tenant spend 50k tokens at 11:59:59 and another 50k at 12:00:00 — the exact burst the limit exists to prevent. This stores one row per request and sums the rows inside the trailing 60 seconds, so the window moves with the clock. Expired rows are evicted on the write path, so the table stays proportional to live traffic with no background job.

Reserve → settle → release. Token cost isn't known until the response arrives, but the decision must be made before the call:

1. reserve() — estimate (prompt + max_tokens), check and insert atomically. Deliberately conservative.

2. settle() — the response's real usage replaces the estimate. A test proves a max_tokens=4000 request settles at ~43 tokens: a tenant is not billed for their own generosity in max_tokens.

3. release() — the call failed, so the reservation is dropped. Our outage does not consume the tenant's budget (also tested).

The race that matters. Read-then-write is not atomic: two concurrent requests both read "40k used of 50k" and both proceed. BEGIN IMMEDIATE takes the write lock before the SELECT. The test fires 50 concurrent reservations of 1000 tokens against a 10000 limit and asserts exactly 10 are allowed.

Why on-disk and not a dict. In-memory state resets every deploy and is not shared between workers — both are how a tenant ends up with 2× their limit. WAL, busy_timeout and synchronous=NORMAL are what make SQLite behave under concurrency; every call is wrapped in asyncio.to_thread so a disk stall never blocks the event loop. A test restarts the limiter against the same file and asserts the quota survived.

A 60k-token request against a 50k limit returns 413, not 429. A 429 would be a lie — no amount of waiting makes it fit. Retry-After is also computed honestly: the limiter walks the window oldest-first to find when enough tokens actually expire, instead of returning a flat 60 and creating a thundering herd at the top of every minute.

Failover policy

Retrying an unretryable error isn't resilience; it's a second way to be slow before returning the same failure.

The timeout is a total deadline. asyncio.timeout(3.0) wraps the whole attempt — DNS, connect, TLS, headers, body. Per-phase HTTP timeouts miss the failure that actually hurts: a provider dribbling bytes just fast enough to keep resetting the read timer while blowing the SLA. Cancelling also tears down the socket, so a timed-out attempt can't hold a pooled connection.

Circuit breaker. After N consecutive failures the primary is skipped entirely for a cooldown, then half-opens for one probe. Without it, every request pays the full 3 s timeout while the primary is down and a provider outage becomes a gateway outage.

Error sanitization

One envelope for every failure:

{"error": {"type": "upstream_error", "code": "gateway.all_providers_failed",
           "message": "No model provider could serve this request. Please retry shortly.",
           "request_id": "f016f46bd42f4fd3afbb940332b46b18"}}

The mock providers return deliberately leaky bodies — db-3.us-east-1.internal, quota-svc-7.internal, org-4417 — and parametrized tests assert none of it appears in any client response. The detail is logged against the same request_id the client received: that id is the join key, so support can answer "what happened to request f016…" without the gateway having leaked anything. A catch-all handler ensures not even a bug in the gateway escapes un-sanitized.

Test summary

$ make test
219 passed in 11.33s

task1_mcp_server      22   stdio purity under a deliberate print(), error-code mapping, 13 malformed-input cases
task2_mcp_gateway     19   authz, batches, notifications, auth failures, sanitization
task3_stream_guardrail 151  every two-way split of a PII string, char-by-char, 200 random chunkings,
                            memory bound over 450 KB, real-socket TTFT
task4_router          27   window eviction, 50-way concurrency race, restart persistence,
                            every failover branch, circuit breaker, leak assertions

What I'd add next, given more time

• Task 1: a refund_id idempotency key so a retried tool call can't double-refund.

• Task 2: JWT/JWKS instead of the static token table, and per-tenant tool allowlists loaded from the control plane rather than a hardcoded admin_ prefix.

• Task 3: redact the request as well as the response (prompts leak PII too), and add a semantic classifier behind the regex layer for the things patterns can't catch (names, addresses).

• Task 4: Redis or a shared store once the gateway runs on more than one node — SQLite is per-node state, which is correct for a single VM and wrong for a fleet. Streaming support in the router, with failover before the first token is committed to the client.