MCPGateway
README.md
# MCPGateway
[](https://github.com/ABHIJEET-MUNESHWAR/MCPGateway/actions/workflows/ci.yml)
[](https://github.com/ABHIJEET-MUNESHWAR/MCPGateway/stargazers)
[](https://github.com/ABHIJEET-MUNESHWAR/MCPGateway/issues)
[](https://github.com/ABHIJEET-MUNESHWAR/MCPGateway/commits/main)
[](./LICENSE)
[](https://www.typescriptlang.org/)
[](https://nodejs.org/)
[](#test-results)
[](https://modelcontextprotocol.io/)
> **A multi-tenant control plane for the Model Context Protocol.** One URL for every
> agent, a deny-by-default policy engine on every tool call, and a replayable audit
> trail tied to a real identity.
---
## Table of contents
- [Why this exists](#why-this-exists)
- [What it does](#what-it-does)
- [Architecture](#architecture)
- [System context](#system-context)
- [Hexagonal layering](#hexagonal-layering)
- [Tool-call flow](#tool-call-flow)
- [Session lifecycle](#session-lifecycle)
- [Component reference](#component-reference)
- [Quick start](#quick-start)
- [Live demo output](#live-demo-output)
- [API](#api)
- [Control plane (GraphQL)](#control-plane-graphql)
- [Data plane (MCP over Streamable HTTP)](#data-plane-mcp-over-streamable-http)
- [Policy model](#policy-model)
- [Resilience](#resilience)
- [Observability](#observability)
- [Data model and partitioning](#data-model-and-partitioning)
- [Performance and complexity](#performance-and-complexity)
- [Test results](#test-results)
- [Configuration](#configuration)
- [Operations](#operations)
- [Project layout](#project-layout)
- [Design decisions](#design-decisions)
- [Roadmap](#roadmap)
- [License](#license)
---
## Why this exists
MCP made it trivial to give an agent tools. It made nothing else trivial.
The moment you run more than one MCP server in production you inherit a pile of
problems the protocol deliberately does not solve:
| Problem | What actually happens without a gateway |
|---|---|
| **N servers, N configs** | Every agent, IDE and teammate re-configures every server by hand. |
| **Credentials everywhere** | Provider API keys sit in plaintext in `claude_desktop_config.json` on laptops. |
| **All-or-nothing access** | An MCP server exposes 40 tools; the agent that needs 1 of them gets all 40. |
| **No identity** | The upstream sees one shared token. "Who deleted the repo?" has no answer. |
| **Silent rug pulls** | A provider changes a tool's schema or description overnight; agents keep calling it. |
| **No blast radius control** | One slow upstream stalls every agent. One buggy agent exhausts a shared quota. |
MCPGateway sits between agents and upstream MCP servers and solves exactly those
six problems, with nothing in the hot path that isn't necessary.
## What it does
- **Aggregates** any number of upstream MCP servers into one namespaced catalog and
exposes it behind a **single connection URL** per agent.
- **Authenticates** every connection to a named `Principal` (a person, an agent, or a
service) inside a `Tenant` — never a shared secret.
- **Authorises** every `tools/list` and `tools/call` against a versioned, deny-by-default
policy that can constrain **individual argument values**, not just tool names.
- **Vaults** upstream credentials with envelope encryption; the agent never sees them,
and rotation is a single mutation.
- **Detects schema drift** by fingerprinting every upstream tool schema and emitting a
domain event the moment a provider changes one behind your back.
- **Wraps every upstream call** in a rate limiter, bulkhead, circuit breaker, retry with
full-jitter backoff, and a timeout — scoped per tenant *and* per provider.
- **Records** every decision and call into a replayable audit trail, and exports
Prometheus metrics for all of it.
---
## Architecture
### System context
```mermaid
flowchart LR
subgraph Agents["AI agents & clients"]
A1["Claude Desktop"]
A2["Custom agent<br/>(LangGraph / SDK)"]
A3["IDE assistant"]
end
subgraph GW["MCPGateway"]
direction TB
DP["Data plane<br/>MCP over Streamable HTTP<br/>/mcp/:token"]
CP["Control plane<br/>GraphQL /graphql"]
POL["Policy engine<br/>deny by default"]
CAT["Catalog<br/>+ drift detection"]
VLT["Credential vault<br/>envelope encryption"]
RES["Resilience layer"]
AUD["Audit + metrics"]
end
subgraph Upstreams["Upstream MCP servers"]
U1["github-mcp"]
U2["slack-mcp"]
U3["payments-mcp"]
end
A1 & A2 & A3 -->|"one URL per identity"| DP
DP --> POL --> CAT --> RES --> U1 & U2 & U3
RES --> VLT
DP --> AUD
CP -.->|"tenants, providers,<br/>policies, tokens, audit"| GW
AUD --> PROM["Prometheus / Grafana"]
```
### Hexagonal layering
Dependencies point strictly inward. The domain has no idea HTTP, GraphQL or Postgres
exist, which is why every rule in it is testable in microseconds.
```mermaid
flowchart TB
subgraph Driving["Driving adapters (inbound)"]
GQL["api/resolvers.ts<br/>GraphQL"]
MCPR["api/mcp-router.ts<br/>MCP JSON-RPC"]
CLI["cli.ts"]
end
subgraph App["Application layer — use cases"]
GS["GatewayService<br/>list + call + intercept"]
CS["CatalogService<br/>aggregate + drift"]
SS["SessionService<br/>tokens + sessions"]
AS["AdminService<br/>control plane"]
end
subgraph Domain["Domain — pure, no I/O"]
POL["policy.ts<br/>evaluatePolicy"]
TEN["tenant.ts"]
PRV["provider.ts"]
SES["session.ts<br/>state machine"]
EVT["events.ts"]
AUD["audit.ts"]
end
subgraph Ports["Ports (interfaces)"]
P["ports.ts<br/>Repositories, CredentialStore,<br/>EventBus, McpClientFactory"]
end
subgraph Driven["Driven adapters (outbound)"]
MEM["adapters/memory<br/>in-memory repos"]
MCPC["mcp/client.ts + transports<br/>http | stdio | in-memory"]
VAULT["app/vault.ts<br/>AES-GCM envelope"]
BUS["app/event-bus.ts"]
end
Driving --> App
App --> Domain
App --> Ports
Ports -.implemented by.-> Driven
```
### Tool-call flow
Every step below is a real code path, in order, with the failure mode it guards.
```mermaid
sequenceDiagram
autonumber
participant A as Agent
participant H as HTTP /mcp/:token
participant S as SessionService
participant G as GatewayService
participant P as Policy engine
participant R as ResiliencePolicy
participant U as Upstream MCP server
participant X as Audit + metrics
A->>H: POST tools/call {name, arguments}
H->>S: authenticate(token)
alt token unknown / revoked / expired / principal disabled
S--xA: 401 unauthorized
end
S-->>H: Caller {tenant, principal}
H->>G: callTool(ctx, request)
G->>G: resolve qualified name -> provider + tool
G->>P: evaluatePolicy(principal, provider, tool, arguments)
alt deny (no matching allow, explicit deny, or argument violation)
P-->>G: Deny {ruleId, reason, violations}
G->>X: record outcome=denied
G--xA: -32010 policy_denied + violations
end
P-->>G: Allow {ruleId}
G->>G: run CallInterceptors (PromptShield hook)
G->>R: execute(scope = tenant/provider)
R->>R: rate limit -> bulkhead -> circuit breaker -> retry -> timeout
R->>U: tools/call with vaulted credential injected
alt upstream fails
U--xR: error
R->>R: retry w/ full jitter, then trip breaker
R-->>G: UpstreamUnavailableError
G->>X: record outcome=error
G--xA: -32020 upstream_unavailable
end
U-->>R: result
R-->>G: result
G->>X: record outcome=success, duration, result digest
G-->>A: tools/call result
```
### Session lifecycle
```mermaid
stateDiagram-v2
[*] --> pending: open(caller, transport)
pending --> ready: initialize handshake
pending --> failed: handshake error
ready --> closing: close(reason)
ready --> failed: transport error
closing --> closed
failed --> closed
closed --> [*]
note right of pending
Illegal transitions are rejected by
a single TRANSITIONS table, not by
scattered if-statements.
end note
```
---
## Component reference
| Component | Path | Responsibility | Key design note |
|---|---|---|---|
| `Result<T, E>` | [src/shared/result.ts](src/shared/result.ts) | Typed success/failure without exceptions | `attempt`/`attemptAsync` quarantine throwing third-party code at the boundary |
| `GatewayError` hierarchy | [src/shared/errors.ts](src/shared/errors.ts) | 14 domain errors carrying `code`, `status`, `retryable`, `details` | One error taxonomy maps cleanly onto both HTTP status and JSON-RPC codes |
| Branded IDs | [src/shared/ids.ts](src/shared/ids.ts) | `TenantId`, `PrincipalId`, `ProviderId`, `SessionId`, … | A `ProviderId` cannot be passed where a `TenantId` is expected — a compile error, not a 3am incident |
| `stableDigest` | [src/shared/ids.ts](src/shared/ids.ts) | Canonical-JSON fingerprint of a tool schema | Key-order independent, so drift detection has no false positives |
| Config | [src/shared/config.ts](src/shared/config.ts) | Zod-validated, nested, parsed once at boot | A bad env var fails at startup, never mid-request |
| Metrics | [src/shared/metrics.ts](src/shared/metrics.ts) | Dependency-free Prometheus counters/gauges/histograms | No client library in the dependency tree; ~200 lines and fully tested |
| `TokenBucketRateLimiter` | [src/resilience/rate-limiter.ts](src/resilience/rate-limiter.ts) | Per-scope quota with lazy refill | `evictIdle` recomputes the refilled value before deciding, so drained buckets are actually reclaimed |
| `CircuitBreaker` | [src/resilience/circuit-breaker.ts](src/resilience/circuit-breaker.ts) | closed → open → half-open | Injected `Clock`, so state-machine tests run instantly and deterministically |
| `Bulkhead` | [src/resilience/bulkhead.ts](src/resilience/bulkhead.ts) | Concurrency cap + bounded queue | Also provides `mapConcurrent` for bounded parallel fan-out |
| `ResiliencePolicy` | [src/resilience/policy.ts](src/resilience/policy.ts) | Composes all five primitives in the correct order | Rate limit **before** bulkhead: reject cheaply before consuming a slot |
| MCP protocol | [src/mcp/protocol.ts](src/mcp/protocol.ts), [src/mcp/jsonrpc.ts](src/mcp/jsonrpc.ts) | Zod-validated JSON-RPC 2.0 + MCP `2025-06-18` | Batch of length 0 is rejected per JSON-RPC §6 |
| Transports | [src/mcp/transports/](src/mcp/transports/) | `http` (Streamable HTTP + SSE), `stdio`, `in-memory` | All three satisfy one `Transport` interface, so tests use in-memory and prod uses HTTP with zero code change |
| `McpRequestRouter<Ctx>` | [src/mcp/server.ts](src/mcp/server.ts) | Generic, context-parameterised MCP server | Same router serves the gateway and the test fixtures |
| Policy engine | [src/domain/policy.ts](src/domain/policy.ts) | Deny-by-default evaluation with argument constraints | Explicit deny always wins; `PolicyBuilder` gives a fluent, typo-proof construction API |
| `CatalogService` | [src/app/catalog-service.ts](src/app/catalog-service.ts) | Aggregate, namespace, diff and refresh upstream tools | Returns a structured `RefreshReport` with `added`/`removed`/`drifted` |
| `GatewayService` | [src/app/gateway-service.ts](src/app/gateway-service.ts) | The hot path: filter, authorise, proxy, record | `CallInterceptor[]` is the extension seam (PromptShield plugs in here) |
| `EnvelopeCredentialStore` | [src/app/vault.ts](src/app/vault.ts) | AES-256-GCM envelope encryption + `secureEquals` | Master key wraps per-record DEKs; comparison is constant-time |
| `UpstreamRegistry` | [src/app/upstream.ts](src/app/upstream.ts) | Pooled upstream clients + per-scope resilience | Evicts a client on error so a poisoned connection is never reused |
| GraphQL API | [src/api/](src/api/) | 12 queries + 14 mutations + 1 subscription | GraphQL over REST: the control plane is a graph, and one round trip beats six |
---
## Quick start
```bash
# Requires Node >= 22 and pnpm
pnpm install
# 1. Watch the whole system work, end to end, with three fake upstream servers.
pnpm demo
# 2. Run the real server (in-memory storage, GraphiQL enabled)
cp .env.example .env
pnpm dev
# GraphQL http://localhost:8080/graphql
# MCP http://localhost:8080/mcp/<connection-token>
# Metrics http://localhost:8080/metrics
# 3. Everything else
pnpm typecheck # tsc over src + test + benchmarks
pnpm test # 213 tests
pnpm test:cov # + coverage gate (85 / 80 / 85 / 85)
pnpm bench # latency + complexity table
pnpm build # dist/
```
With Docker:
```bash
export VAULT_MASTER_KEY="$(openssl rand -base64 32)"
export ADMIN_API_KEY="$(openssl rand -hex 32)"
docker compose up -d # gateway + postgres
docker compose --profile monitoring up -d # + prometheus, grafana, alertmanager, otel
```
---
## Live demo output
`pnpm demo` boots three in-process MCP servers, registers them for tenant `acme`,
publishes a policy, and then exercises the whole system. This is **verbatim output**,
not a mock-up:
```text
1. Aggregated catalog: three upstream MCP servers behind one endpoint
github__create_issue <- github/create_issue digest=c89537e73a42
github__list_repos <- github/list_repos digest=e04ab9142728
slack__post_message <- slack/post_message digest=4c58eebbbfd6
payments__refund <- payments/refund digest=860462b4e7dc
2. Per-identity tool visibility (deny by default)
alice sees 3: github__create_issue, github__list_repos, slack__post_message
releasebot sees 1: github__create_issue
3. Allowed call, proxied to the real upstream server
result: [{"type":"text","text":"issue opened in acme/api: Flaky integration test"}]
4. Argument-level policy: the agent may only touch approved repos
allowed -> [{"type":"text","text":"issue opened in acme/web: Release 1.4.0 checklist"}]
denied -> policy_denied: policy denied: argument constraints violated
violations=["repo must be one of [\"acme/api\",\"acme/web\"]"]
5. Explicit deny always wins, even for a privileged user
denied -> policy denied: refunds require a finance-approved principal
6. Upstream schema drift detection
refreshed 3 providers, drift events: 1
DRIFT github__create_issue: c89537e7 -> 3fea86ad
7. Replayable audit trail tied to real identity
principal tool outcome duration
alice payments__refund denied 0ms
releasebot github__create_issue success 0ms
releasebot github__create_issue denied 0ms
alice github__create_issue success 0ms
8. Prometheus metrics (excerpt)
mcpgateway_tool_calls_total{outcome="success",provider="github",tool="github__create_issue"} 2
mcpgateway_tool_calls_total{outcome="denied",provider="github",tool="github__create_issue"} 1
mcpgateway_tool_calls_total{outcome="denied",provider="payments",tool="payments__refund"} 1
mcpgateway_policy_decisions_total{effect="allow",tool="github__create_issue"} 2
mcpgateway_policy_decisions_total{effect="deny",tool="github__create_issue"} 1
mcpgateway_policy_decisions_total{effect="deny",tool="payments__refund"} 1
```
Step 4 is the point of the whole project: `releasebot` is allowed to call
`github__create_issue`, but only with `repo` in an approved set. Tool-level
allow-lists cannot express that. This one can.
---
## API
### Control plane (GraphQL)
Guarded by `x-api-key: $ADMIN_API_KEY`. GraphiQL is served at `/graphql` when
`GRAPHIQL_ENABLED=true` (never in production).
<details>
<summary><strong>Queries</strong></summary>
| Query | Purpose |
|---|---|
| `tenants` / `tenant(id)` | Tenants with nested principals and providers |
| `principals(tenantId)` | Identities in a tenant |
| `providers(tenantId)` / `provider(id)` | Registered upstream MCP servers |
| `providerHealth(providerId)` | Status, tool count, last refresh, last error |
| `catalog(tenantId, providerIds)` | The aggregated, namespaced tool catalog |
| `policy(tenantId)` | The active policy document with its version |
| `evaluatePolicy(...)` | **Dry-run a decision** before shipping a policy change |
| `sessions(tenantId)` / `connections(tenantId)` | Live sessions and issued tokens |
| `audit(filter)` | Paginated, filterable audit trail |
| `resilience` | Live circuit-breaker / bulkhead state per scope |
</details>
<details>
<summary><strong>Mutations</strong></summary>
| Mutation | Purpose |
|---|---|
| `createTenant` / `setTenantPlan` / `setTenantDisabled` | Tenant lifecycle |
| `createPrincipal` / `setPrincipalDisabled` | Identity lifecycle |
| `registerProvider` / `setProviderDisabled` | Upstream lifecycle (secret stored on registration) |
| `rotateCredential` | Re-wrap an upstream secret without touching the provider record |
| `refreshCatalog` | Force re-discovery; returns per-provider diffs including drift |
| `putPolicy` | Publish a new **version** of the access policy |
| `issueConnection` / `revokeConnection` | Mint and kill the single per-agent MCP URL |
| `closeSession` | Terminate a live agent session |
</details>
<details>
<summary><strong>Subscription</strong></summary>
`events(tenantId)` streams the `DomainEvent` union — tool calls, policy decisions,
schema drift, credential rotations, session transitions — filtered to one tenant.
</details>
**Dry-run a policy change before it reaches production:**
```graphql
query {
evaluatePolicy(
tenantId: "acme"
principalId: "releasebot"
providerId: "github"
toolName: "create_issue"
arguments: { repo: "acme/secret-infra", title: "..." }
) {
effect # "deny"
ruleId # "ci-can-open-issues"
reason # "argument constraints violated"
violations # ["repo must be one of [\"acme/api\",\"acme/web\"]"]
}
}
```
### Data plane (MCP over Streamable HTTP)
`POST /mcp/:connectionToken` speaks JSON-RPC 2.0. The token *is* the identity — no
extra header, so it drops straight into any MCP client config:
```json
{
"mcpServers": {
"everything": {
"url": "https://gateway.example.com/mcp/mcp_live_a1b2c3d4"
}
}
}
```
| Method | Behaviour |
|---|---|
| `initialize` | Negotiates protocol version, returns capabilities |
| `tools/list` | Returns **only** the tools this principal is allowed to call |
| `tools/call` | Policy check → interceptors → resilient proxy → audit |
| `resources/list`, `prompts/list` | Proxied and aggregated |
| `ping` | Liveness |
| batch `[...]` | Supported; empty arrays rejected per JSON-RPC §6 |
`GET /mcp/:token` with `accept: text/event-stream` opens the server→client SSE stream.
JSON-RPC error codes: `-32010` policy denied, `-32020` upstream unavailable,
`-32021` upstream timeout, `-32030` rate limited — alongside the standard
`-32600`/`-32601`/`-32602`/`-32603`.
---
## Policy model
A policy is a **versioned, ordered list of rules**. Evaluation is deny-by-default:
1. If any `deny` rule matches → **deny**. Explicit deny always wins.
2. Otherwise, if an `allow` rule matches → check its argument constraints.
3. Otherwise → **deny** (no rule matched).
```typescript
new PolicyBuilder(TenantId("acme"), version)
.allow({
id: "eng-full-access",
subjects: ["group:engineering"],
providers: ["github", "slack"],
tools: ["*"],
})
.allow({
id: "ci-can-open-issues",
subjects: ["group:ci"],
providers: ["github"],
tools: ["create_issue"],
arguments: [
{ path: "repo", rule: { kind: "one_of", values: ["acme/api", "acme/web"] } },
{ path: "title", rule: { kind: "max_length", value: 120 } },
],
})
.deny({
id: "no-refunds",
subjects: ["*"],
providers: ["payments"],
tools: ["refund"],
description: "refunds require a finance-approved principal",
})
.build();
```
Subjects match a principal id (`alice`), a group (`group:engineering`), a kind
(`kind:agent`) or `*`. Argument constraints address nested values by dotted path
(`config.target.region`) and support `one_of`, `not_one_of`, `matches` (regex),
`max_length`, `max_items`, `range` and `required`.
---
## Resilience
Every upstream call passes through five primitives, composed in this order and scoped
to `tenant/provider` so one noisy tenant cannot affect another:
```mermaid
flowchart LR
C["callTool"] --> RL["Rate limiter<br/>token bucket"]
RL -->|"reject cheaply"| E1["rate_limited (-32030)"]
RL --> BH["Bulkhead<br/>concurrency + queue"]
BH -->|"shed load"| E2["bulkhead_rejected"]
BH --> CB["Circuit breaker<br/>closed/open/half-open"]
CB -->|"fail fast"| E3["circuit_open"]
CB --> RT["Retry<br/>full-jitter backoff"]
RT --> TO["Timeout"]
TO --> U["Upstream"]
```
Ordering is deliberate: rate limiting is O(1) and rejects before a bulkhead slot is
consumed; the breaker sits *outside* retry so a dead upstream is not retried 3× per
request while it is already known to be down; the timeout is innermost so it bounds a
single attempt rather than the whole retry sequence.
Only errors marked `retryable` are retried — a `policy_denied` or `validation_failed`
is never retried, because it will never succeed.
---
## Observability
**Structured logging** (`pino`) with correlation IDs threaded from the inbound request
through every service call. `redactArguments` strips tool arguments and secret-shaped
fields before anything is written.
**Metrics** at `GET /metrics`:
| Metric | Type | Labels |
|---|---|---|
| `mcpgateway_tool_calls_total` | counter | `tenant`, `provider`, `tool`, `outcome` |
| `mcpgateway_tool_call_duration_ms` | histogram | `provider`, `tool` |
| `mcpgateway_policy_decisions_total` | counter | `tenant`, `tool`, `effect` |
| `mcpgateway_upstream_requests_total` | counter | `provider`, `outcome` |
| `mcpgateway_upstream_duration_ms` | histogram | `provider`, `method` |
| `mcpgateway_resilience_events_total` | counter | `scope`, `event` |
| `mcpgateway_schema_drift_total` | counter | `provider` |
| `mcpgateway_credential_operations_total` | counter | `operation`, `outcome` |
| `mcpgateway_active_sessions` | gauge | `tenant`, `transport` |
| `mcpgateway_catalog_tools` | gauge | `tenant`, `provider` |
| `mcpgateway_graphql_requests_total` | counter | `operation`, `outcome` |
**Dashboards and alerts** ship in [monitoring/](monitoring/): a Grafana overview
dashboard (RED + governance + resilience rows), 12 Prometheus alert rules, an
Alertmanager route into Grafana OnCall with severity-based escalation and inhibition,
and an OpenTelemetry collector config that **deletes tool arguments and auth headers**
before any trace leaves the process.
---
## Data model and partitioning
See [db/migrations/0001_init.sql](db/migrations/0001_init.sql).
Every table is keyed by `tenant_id` first — that single column is the shard key, so a
future horizontal split moves whole tenants and never splits a tenant's rows.
`tool_call_audit` is the only unbounded table, so it is:
- **RANGE partitioned by month** on `occurred_at` — retention becomes `DROP TABLE` of a
whole partition (O(1)) instead of a multi-million-row `DELETE`, and every
time-bounded audit query prunes to a single partition;
- **HASH sub-partitioned 4 ways by `tenant_id`** within each month, so one high-volume
tenant cannot make another tenant's index scans hot.
`create_audit_partition()` and `drop_audit_partitions_older_than()` are shipped as
functions; the current month plus three ahead are pre-created so ingestion never hits a
missing partition.
Domain events go through a **transactional outbox** (`event_outbox`) written in the same
transaction as the state change — no dual-write, no lost events.
Connection tokens are stored **hashed**; a database leak yields no working credentials.
---
## Performance and complexity
`pnpm bench` — measured on Node v24.10.0, linux/x64:
| Operation | Complexity | Iterations | p50 | p95 | p99 | ops/sec |
|---|---|---|---|---|---|---|
| `policy.evaluate` (3 rules) | O(R × (S+P+T+A)) | 50,000 | 0.0007 ms | 0.0012 ms | 0.0031 ms | **941,355** |
| `policy.evaluate` (201 rules) | O(R × (S+P+T+A)) | 20,000 | 0.0260 ms | 0.0492 ms | 0.1991 ms | 30,536 |
| `stableDigest` (schema fingerprint) | O(n log n) on keys | 20,000 | 0.0052 ms | 0.0078 ms | 0.0204 ms | 144,680 |
| `rateLimiter.tryAcquire` | O(1) amortised | 200,000 | 0.0002 ms | 0.0003 ms | 0.0005 ms | **3,914,955** |
| `catalog.resolve` | O(1) | 20,000 | 0.0003 ms | 0.0026 ms | 0.0031 ms | 1,862,237 |
| `gateway.listTools` (policy-filtered) | O(T × R) | 5,000 | 0.0051 ms | 0.0087 ms | 0.0412 ms | 125,513 |
| `gateway.callTool` (end-to-end) | O(R + upstream) | 3,000 | 0.0450 ms | 0.1473 ms | 0.3507 ms | 14,615 |
| `audit.query` | O(n) over tenant slice | 2,000 | 0.4546 ms | 0.7978 ms | 1.1650 ms | 1,931 |
| `metrics.render` | O(series) | 2,000 | 0.0168 ms | 0.0273 ms | 0.1132 ms | 46,864 |
Where R = rules, S/P/T = subject/provider/tool patterns per rule, A = argument
constraints, T = tools in catalog, n = records.
**Reading the numbers.** Policy evaluation adds **≈0.7 µs** to a call that will spend
tens of milliseconds in the upstream — governance is free at this scale. Catalog
resolution is a `Map` lookup, deliberately: the hot path must not scan. The 201-rule
case is the honest worst case and is the one place a future optimisation (indexing
rules by provider before scanning) would pay off; it is not needed yet, and the
benchmark exists so a regression is visible the day it lands.
`audit.query` is the slowest operation by design — it is an operator-facing scan, not a
hot path, and in Postgres it is served by the partitioned index rather than this
in-memory adapter.
---
## Test results
`pnpm test:cov` — **213 tests across 14 files, all passing.**
| Suite | Tests | What it pins down |
|---|---|---|
| [test/shared/result.test.ts](test/shared/result.test.ts) | 8 | `Result` combinators, `attempt`/`attemptAsync` around throwing code |
| [test/shared/errors.test.ts](test/shared/errors.test.ts) | 5 | Error taxonomy, `toGatewayError` on unknown throwables, retryability |
| [test/shared/ids.test.ts](test/shared/ids.test.ts) | 6 | Branding, `hashToken`, key-order-independent `stableDigest` |
| [test/shared/metrics.test.ts](test/shared/metrics.test.ts) | 10 | Counter/gauge/histogram maths, label escaping, exposition format |
| [test/shared/config.test.ts](test/shared/config.test.ts) | 6 | Env mapping, defaults, rejection of malformed values |
| [test/resilience/primitives.test.ts](test/resilience/primitives.test.ts) | 30 | Timeout, retry jitter bounds, breaker transitions, bucket refill, bulkhead queueing, full policy composition — all on a `ManualClock` |
| [test/domain/policy.test.ts](test/domain/policy.test.ts) | 16 | Deny-by-default, explicit-deny precedence, wildcards, every argument constraint, nested paths |
| [test/domain/session.test.ts](test/domain/session.test.ts) | 10 | Legal and illegal state transitions |
| [test/app/vault-events.test.ts](test/app/vault-events.test.ts) | 18 | Envelope encrypt/decrypt, rotation, tamper detection, constant-time compare, event bus fan-out and handler isolation |
| [test/mcp/protocol.test.ts](test/mcp/protocol.test.ts) | 22 | JSON-RPC framing, batch rules, version negotiation, router dispatch and error-code mapping |
| [test/mcp/transports.test.ts](test/mcp/transports.test.ts) | 17 | Streamable HTTP incl. SSE multi-event, 202/204, bad content-type, network failure; stdio against a real child process; in-memory isolation |
| [test/app/services.test.ts](test/app/services.test.ts) | 28 | Catalog aggregation and drift diffs, gateway allow/deny/upstream-failure paths, session lifecycle, admin mutations |
| [test/api/http.test.ts](test/api/http.test.ts) | 21 | Health, metrics, admin auth, MCP routing, error mapping |
| [test/api/graphql.test.ts](test/api/graphql.test.ts) | 16 | JSON scalar over every literal kind, all queries, all mutations, subscription tenant-filtering |
Coverage (thresholds: statements 85 / branches 80 / functions 85 / lines 85):
| Module | Stmts | Branch | Funcs |
|---|---|---|---|
| `src/shared` | 99.53% | 91.63% | 97.11% |
| `src/domain` | 98.97% | 96.52% | 100% |
| `src/resilience` | 96.55% | 88.18% | 91.66% |
| `src/app` | 94.07% | 84.50% | 94.73% |
| `src/mcp` | 91.71% | 84.12% | 82.22% |
| `src/api` | 82.77% | 77.41% | 82.45% |
| **All files** | **92.47%** | **86.38%** | **91.46%** |
The domain and resilience layers — where a bug is silent and expensive — are near
100%. The API layer is lower because its uncovered branches are transport plumbing
already exercised end-to-end by the HTTP suite.
**Six real source bugs were found by these tests**, not by review:
1. `evictIdle()` compared a *stale* token count against the burst size, so a bucket that
had been drained once was never reclaimed — a slow memory leak per idle scope.
2. `rpcCodeFor` was not exported, so callers silently fell back to `-32603`.
3. Zod parse failures mapped to `-32603` (internal error) instead of `-32602`
(invalid params) — the client could not tell "you sent bad arguments" from "we broke".
4. An unknown MCP method threw an anonymous `Error` with a stapled property that
`rpcCodeFor` ignored, so `method not found` reported as an internal error.
5. `JsonRpcPayloadSchema` accepted an empty batch `[]`, which JSON-RPC 2.0 §6 forbids.
6. `InMemoryTransport` threw a plain `Error` on a closed transport, so callers could not
treat transports uniformly — it now throws `UpstreamUnavailableError` like the others.
---
## Configuration
Every value is parsed and validated once at startup by
[src/shared/config.ts](src/shared/config.ts); see [.env.example](.env.example) for the
full annotated list.
| Group | Variables |
|---|---|
| HTTP | `HTTP_HOST`, `HTTP_PORT`, `HTTP_REQUEST_TIMEOUT_MS`, `HTTP_MAX_BODY_BYTES` |
| Storage | `STORAGE_DRIVER` (`memory`\|`postgres`), `DATABASE_URL`, `DATABASE_POOL_SIZE` |
| Secrets | `VAULT_MASTER_KEY`, `VAULT_ROTATION_DAYS`, `ADMIN_API_KEY`, `GRAPHIQL_ENABLED` |
| Resilience | `RESILIENCE_TIMEOUT_MS`, `RESILIENCE_RETRY_ATTEMPTS`, `RESILIENCE_RETRY_BASE_DELAY_MS`, `RESILIENCE_RETRY_MAX_DELAY_MS`, `RESILIENCE_CIRCUIT_FAILURE_THRESHOLD`, `RESILIENCE_CIRCUIT_RESET_MS`, `RESILIENCE_RATE_LIMIT_PER_MINUTE`, `RESILIENCE_BULKHEAD_CONCURRENCY` |
| Catalog | `CATALOG_REFRESH_INTERVAL_MS`, `CATALOG_NAMESPACE_SEPARATOR`, `CATALOG_MAX_TOOLS_PER_TENANT` |
| Audit | `AUDIT_RETENTION_DAYS`, `AUDIT_RECORD_PAYLOADS`, `AUDIT_MAX_PAYLOAD_BYTES` |
| Observability | `METRICS_ENABLED`, `OTEL_EXPORTER_OTLP_ENDPOINT` |
> `VAULT_MASTER_KEY` and `ADMIN_API_KEY` have development defaults so `pnpm demo` runs
> with zero setup. `docker-compose.yml` refuses to start without real values.
## Operations
**Runbook — first three things to check.**
| Alert | Likely cause | First action |
|---|---|---|
| `UpstreamSchemaDrift` | A provider silently changed a tool definition (the MCP rug-pull vector) | `refreshCatalog` and read the `drifted` diff before letting agents keep calling it |
| `CircuitBreakerOpen` | An upstream is down or slow | `query { resilience { scope state } }`, then `providerHealth(providerId)` for `lastError` |
| `PolicyDenySpike` | A policy change broke a legitimate workflow, or an agent is misbehaving | `audit(filter: {outcome: denied})` and group by `principalId` |
| `RateLimitSaturation` | A tenant is over quota | `setTenantPlan`, or raise `RESILIENCE_RATE_LIMIT_PER_MINUTE` |
| `ToolCallLatencyP99High` | Upstream latency, not gateway latency | Compare `mcpgateway_upstream_duration_ms` against `mcpgateway_tool_call_duration_ms` |
**Incident containment.** `revokeConnection(token)` kills one agent instantly;
`setPrincipalDisabled` kills every session for an identity; `setProviderDisabled`
removes an upstream from every catalog without deleting its configuration.
**Container.** Multi-stage build → distroless, non-root uid 10001, read-only root
filesystem, all capabilities dropped, `no-new-privileges`, and a `HEALTHCHECK` that
runs `mcpgateway healthcheck` (no shell or curl needed in the image).
---
## Project layout
```text
MCPGateway/
├── src/
│ ├── shared/ Result, errors, branded ids, clock, metrics, logger, config
│ ├── resilience/ timeout, retry, circuit breaker, rate limiter, bulkhead, policy
│ ├── mcp/ JSON-RPC + MCP protocol, client, generic server router
│ │ └── transports/ http (streamable + SSE) | stdio | in-memory
│ ├── domain/ tenant, provider, session, policy, events, audit (pure, no I/O)
│ ├── app/ ports + CatalogService, GatewayService, SessionService,
│ │ AdminService, vault, upstream registry, event bus
│ ├── adapters/ memory storage, MCP client factory, fixture MCP server
│ ├── api/ GraphQL SDL + resolvers, MCP router, HTTP app
│ ├── demo/ the runnable end-to-end scenario
│ ├── container.ts composition root
│ ├── server-entry.ts
│ └── cli.ts
├── test/ 213 tests mirroring the src layout
├── benchmarks/ criterion-style latency + complexity harness
├── db/migrations/ partitioned schema
├── monitoring/ prometheus, alerts, alertmanager, otel, grafana
├── postman/ runnable collection with assertions
└── .github/workflows/ ci.yml
```
## Design decisions
**Why GraphQL for the control plane and JSON-RPC for the data plane?** They are
different problems. The control plane is a graph — "show me this tenant, its
principals, their providers, and the last 20 audit records" is one query instead of six
REST round trips, and the subscription gives operators a live event feed for free. The
data plane is not negotiable: MCP *is* JSON-RPC 2.0, so the gateway speaks it exactly.
**Why a hand-written metrics implementation?** It is ~200 lines, fully tested, has zero
dependencies, and emits the exposition format Prometheus expects. Pulling in a client
library to count integers is a dependency I would have to defend at audit time.
**Why branded IDs?** Passing a `ProviderId` where a `TenantId` belongs is the kind of
bug that reaches production and leaks data across tenants. Branding makes it a compile
error. Same reasoning behind the `TRANSITIONS` table for sessions and the discriminated
`DomainEvent` union — the type checker enforces exhaustiveness so a new event type
cannot be silently ignored.
**Why interfaces for everything, in-memory for now?** The ports in
[src/app/ports.ts](src/app/ports.ts) are the contract; `adapters/memory` is one
implementation and a Postgres one is another. That is why the entire test suite runs in
~2 seconds with no containers, and why the schema in `db/migrations/` can land without
touching a single line of domain code.
**Why a `CallInterceptor` seam?** Prompt-injection scanning, PII redaction and
exfiltration detection are a separate concern with a separate release cadence. They
belong behind an interface, not inlined into `GatewayService`.
## Roadmap
- [ ] Postgres adapter behind the existing ports (schema is already written)
- [ ] OAuth 2.1 device flow for upstream providers that require user consent
- [ ] SAML / SCIM for enterprise identity sync
- [ ] Policy simulation against replayed historical traffic
- [ ] PromptShield integration through the `CallInterceptor` seam
## License
[MIT](./LICENSE)
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues