# wardrobe-mcp

[![CI](https://github.com/pravoobi/wardrobe-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/pravoobi/wardrobe-mcp/actions/workflows/ci.yml)

A **self-hosted MCP server** that gives Alexa+ a customer's wardrobe: what they own, what
they've saved, what they could buy — and, the differentiator, what any of it **looks like
on them**, rendered by the existing on-device virtual try-on pipeline
([`pravoobi/try-on`](https://github.com/pravoobi/try-on)).

> "Alexa, I have a sangeet on Saturday. What should I wear? Budget eight thousand."
> → `suggest_outfits` reads out 3 options with reasons → "Show me the magenta one on me."
> → `preview_look` returns a rendered try-on on the Echo Show → "Save that." → "Compare them."
> → "Go with the magenta." → `record_reaction(love)` → Alexa+ offers the purchase link.

**Thesis:** Alexa+ is the stylist's *voice*; this server is the stylist's *memory and eyes*.
Every tool is a thin, fast, structured lookup. Anything slow (rendering) is pre-computed.

Built for the Amazon *Build, Ship, Shape* hackathon — **Alexa+** track, plus the **AWS
Builder** and **Open Source** mini challenges. MIT licensed.

![The sangeet flow in the simulated Alexa+ web app: suggest, preview, save, compare](docs/demo.gif)

*Recorded 2026-09-08 against a locally-seeded server with `RENDER_MODE=browser` — every
preview is the real try-on pipeline compositing the garment onto the model photo, not the
`local`-mode fallback (garment shown beside the photo). The "Bedrock is unavailable"
notices are real and honest, not staged — the account-verification blocker in
[`docs/aws.md`](docs/aws.md) was still open at recording time, so `src/web/ruleReasoner.ts`
answered instead of real Bedrock. Same real MCP tools either way.*

## Try it in a minute

Everything below is the **live deployed server** (AWS App Runner, `us-east-1`) — no signup,
no API key, nothing to install.

1. **[Open the simulated Alexa+ app →](https://ba93fetekm.us-east-1.awsapprunner.com/simulator)**
   Click **Connect** → **Continue as Praveen** (a real OAuth 2.1 PKCE login against this
   server's own `/oauth/*`), then click the first suggested line:
   *"I have a sangeet on Saturday. What should I wear? Budget eight thousand."*
   Keep clicking down the list — suggest → preview → save → another → save → compare.
   Every grey `TOOL CALL` bubble is a real MCP `tools/call` over Streamable HTTP, and every
   image on the Echo Show panel is served from CloudFront.
2. **[See the OAuth discovery document →](https://ba93fetekm.us-east-1.awsapprunner.com/.well-known/oauth-authorization-server)**
   — `S256`, the `resource` parameter, no Dynamic Client Registration: what Alexa+ requires.
3. **[Check which commit is live →](https://ba93fetekm.us-east-1.awsapprunner.com/version)**
   — the deployed `GIT_SHA`, baked in at image build. It tracks the last commit that
   changed the *server*; docs-only commits move `main` ahead without a redeploy.

Prefer to drive it as a raw MCP client? `POST /mcp` unauthenticated returns the `401`
shape Alexa+ expects; the full OAuth + `tools/call` flow is in [Auth](#auth-oauth-21).

## Architecture

```
  Alexa+ (MCP client) ──Streamable HTTP + Bearer──► wardrobe-mcp (Node 22 + TS)
                                                     ├─ /mcp        MCP server (tools, resources, prompts)
                                                     ├─ /oauth/*    OAuth 2.1 AS (PKCE S256, resource param)
                                                     ├─ /.well-known/oauth-*   discovery
                                                     └─ /renders/*  rendered looks (dev; CloudFront in prod)
                          store/   SQLite (dev) │ DynamoDB (prod)
                          render/  queue → Playwright worker → drives try-on's own WebMCP tools
```

**The renderer bridge:** the try-on pipeline is browser-only (LiteRT.js, WebGPU/Wasm). We
don't port it. The render worker launches headless Chromium, opens the deployed try-on app,
and calls the page's **own WebMCP tools** via
`document.modelContext.tools.get("apply_tryon").execute(...)`, then snapshots the result
canvas. One tool contract, two transports. Rendering is never in the request path —
`preview_look` / `compare_looks` are cache reads; a miss enqueues and returns
`{ status: "rendering" | "composing", retryAfterSeconds }`.

Two render modes (`RENDER_MODE`):

| mode | what it does | when |
|---|---|---|
| `local` *(default)* | composes a look card with `sharp` from the flat garment PNG + the customer photo — fast, offline, deterministic | dev, tests, demo |
| `browser` | the real bridge above — drives the deployed try-on app in headless Chromium | recording the demo, eyeballing real try-on output |
| `stub` | solid-colour placeholder | unit tests |

`browser` mode needs Playwright's **full** Chromium (`channel: "chromium"`), not the
default headless shell — the shell ships no GPU stack at all, so `requestAdapter()`
returns `null` and the try-on app waits forever on a WebGPU device that never arrives.
With the right binary it's ~13 s per render, not "slow because of CPU fallback" (that was
our own wrong diagnosis for a while — see [`docs/friction-log.md`](docs/friction-log.md)
#4a/#4b for the full trail). It stays opt-in and `local` stays the default because
production runs on a 0.25 vCPU / 0.5 GB App Runner instance, which is too small to run
Chromium plus an ML pipeline safely — not because it doesn't work.

## Tool surface

| Tool | Purpose |
|---|---|
| `list_wardrobe` | Clothes the customer owns/saved (filter by category / occasion / colour). |
| `search_catalog` | Ranked catalog search — occasion, colour, category, budget, free text. |
| `suggest_outfits` | Up to 3 outfits for an occasion, **deterministically ranked** (no LLM). |
| `preview_look` | A garment rendered on the customer's photo. Cache read. |
| `save_look` | Persist a look with a label. |
| `list_looks` | The looks saved this session. |
| `compare_looks` | 2–3 saved looks side by side. |
| `record_reaction` | `love` / `like` / `try_another` / `reject` → guidance for the next turn. |
| `add_to_wardrobe` | Add a catalog garment; triggers background pre-render. |
| `plan_week` | Up to a week of events, one outfit each, never repeating a garment. |

Resources: `wardrobe://catalog`, `wardrobe://looks/{lookId}`. Prompt: `style_me`.

## Run it locally

```bash
npm install
cp .env.example .env
npm run seed      # demo user "Praveen" + try-on test photo + whole-catalog pre-render
npm run dev       # http://localhost:3000/mcp
```

`npm run seed` fetches the catalog garment PNGs and a test photo from the deployed try-on
app (cached under `data/assets/`). Render one look to eyeball it:
`tsx scripts/render-one.ts dress-magenta-wrap-01 photo-01` (add `--browser` for the real
bridge).

**Or one command, no Node install needed:**

```bash
docker compose up --build          # http://localhost:3000
docker compose run --rm seed       # once, in another terminal — demo user + pre-rendered looks
```

SQLite + local renders persist on the host under `./data` and `./renders` (both
gitignored), so a later `docker compose up` picks up where you left off. `seed` uses the
`build` image stage (has `tsx` + `scripts/`) rather than the slim production one.

## Auth (OAuth 2.1)

A minimal OAuth 2.1 authorization server runs in the same process — exactly what Alexa+
expects, nothing it rejects:

- authorization-code + PKCE **S256 only**; RFC 8707 `resource` mandatory, must equal the
  canonical MCP URL; public pre-registered clients (no Dynamic Client Registration)
- `GET /.well-known/oauth-authorization-server` + `/.well-known/oauth-protected-resource`
- `GET /oauth/authorize` → `/link` demo consent → `POST /oauth/token` (15-min access,
  30-day refresh, rotating; all persisted hashed)
- unauthenticated `POST /mcp` → **`401` with no `WWW-Authenticate`** (Amazon's quirk)

Ops: `GET /healthz` (add `?deep=1` to also touch the store — DynamoDB/SQLite — rather
than just confirm the process is up), `GET /version` (package version + `GIT_SHA`, so
it's obvious which commit a deployed instance is actually running — see friction log
#11, a stale image was the whole bug). `/link`, `/oauth/token`, `/api/simulate/turn` and
`/mcp` are all rate-limited per IP (`src/auth/rateLimit.ts`); graceful shutdown on
`SIGTERM`/`SIGINT` so an App Runner deploy doesn't sever in-flight requests.

Dev shortcut: `Authorization: Bearer dev_praveen` works when `ALLOW_DEV_TOKENS=true`
(auto-off when `NODE_ENV=production`). Point MCP Inspector at `http://localhost:3000/mcp`
and it runs the full PKCE dance.

Expose it for Alexa+: `cloudflared tunnel --url http://localhost:3000`. When you do,
restart the dev server with `MCP_CANONICAL_URL` set to the tunnel's `https://…` hostname
(the OAuth `resource` param must match it exactly) and `ALLOW_DEV_TOKENS=false` (a
publicly reachable instance shouldn't accept `dev_*` bearer tokens). Verified
2026-09-07: real OAuth 2.1 PKCE + `tools/call` round trip in ~150 ms through a quick
tunnel — handy for local-machine testing; the real deployed instance is
`https://ba93fetekm.us-east-1.awsapprunner.com` (see `docs/aws.md`).

## Verify

```bash
npm run typecheck
npm test            # unit + OAuth + discovery + CORS + Store contract + e2e MCP client
                    # 86 pass; 9 DynamoDB-Local tests skip unless `npm run test:dynamo`
npm run smoke       # boots the server, runs OAuth + the full sangeet flow
npm run test:dynamo # Store contract against DynamoDB Local (needs Docker)
```

`npm run smoke` asserts: unauthenticated `POST /mcp` → `401` **without** `WWW-Authenticate`;
discovery advertises `S256`; a full PKCE loop yields a token that works on `/mcp`;
negotiated protocol ≥ `2025-11-25`; every image tool a cache hit < 100 ms.

## Status

- [x] **Phase 1** — skeleton MCP over Streamable HTTP, catalog + wardrobe + suggest tools, SQLite store, smoke flow.
- [x] **Phase 2** — renderer bridge (`local` sharp compositor + opt-in `browser` bridge to the try-on app's WebMCP tools), `sharp` compare sheets, cache + queue, whole-catalog pre-render on seed, inline image blocks. Catalog synced from `pravoobi/try-on`. Tool schemas frozen.
- [x] **Phase 3** — OAuth 2.1 AS (PKCE S256, `resource`, refresh rotation, every rejection-path tested); DynamoDB `Store` (single-table, parity suite vs SQLite, verified against DynamoDB Local); S3 + CloudFront render storage; Bedrock garment tagger (feature-flagged); `Dockerfile` (Playwright base) + `apprunner.yaml`.
- [x] **Phase 3 (ops)** — ECR image pushed, DynamoDB table + S3 bucket + IAM roles live in
  `us-east-1`; Bedrock model access granted. The account-verification hold (friction log #9)
  was resolved 2026-10-14 via an AWS Support exception request — App Runner, CloudFront, and
  Bedrock standard-tier models are all live. **Deployed**: `https://ba93fetekm.us-east-1.awsapprunner.com`,
  images served over a real CloudFront distribution (`dyswjdrhfb962.cloudfront.net`). Confirmed
  live: unauthenticated `POST /mcp` → `401` with no `WWW-Authenticate`; discovery doc reports
  the correct issuer. **Full authenticated round trip verified** — real OAuth 2.1 PKCE login,
  real `@modelcontextprotocol/sdk` client, the whole sangeet sequence against the live service,
  found and fixed 3 real deploy-only bugs along the way (stale image missing `plan_week`,
  a `RENDER_MODE` cache-key mismatch, a missing `s3:ListBucket` IAM grant that broke
  `compare_looks` on any true cache miss — see `docs/aws.md`). **Latency measured from a
  real us-east-1 CloudShell**: `suggest_outfits` 90 ms (37–50 ms on repeat), `preview_look`
  (cache hit) 127–152 ms, `compare_looks` (cache hit) 123 ms — all well under the 500 ms
  budget. Phase 3 is fully done.
- [x] ~~**Phase 4** — Alexa+ add-on (`alexa-ai deploy`).~~ Skipped: the add-ons console is
  partner-gated ("coming soon"), not self-serve. Pivoted to Phase 5 (Devpost's officially
  sanctioned simulated-experience path — see friction log #8).
- [x] **Phase 5** — simulated Alexa+ web app (`web/simulator.html`): real OAuth 2.1 PKCE
  login + the real `@modelcontextprotocol/sdk` client (bundled for the browser, no CDN) over
  Streamable HTTP. **The full sangeet flow runs end-to-end in a real browser**: `suggest_outfits`
  → `preview_look` → `save_look` → `preview_look` (another option) → `save_look` →
  `compare_looks`, with real rendered images on the simulated Echo Show screen. Bedrock (the
  reasoning step that picks which tool to call) is still blocked account-wide (friction log
  #9), so `src/web/ruleReasoner.ts` — a small deterministic intent matcher — stands in
  automatically, tagged and visibly labeled in the transcript, never silent. It reverts to
  real Bedrock the moment the account clears, no config to flip back.

The full sangeet flow (`npm run smoke`) runs end-to-end from a cold seed: `suggest_outfits`
→ `preview_look` → `save_look` → `compare_looks` → `record_reaction`, every image tool a
cache hit under 100 ms.

## Deploy to AWS

`STORE=dynamodb STORAGE=s3 TAGGING_PROVIDER=bedrock` in `us-east-1`. Container from the
Playwright base image (`Dockerfile`), run on App Runner (`apprunner.yaml`) with
`min instances = 1`. Full steps + the single-table schema + IAM notes in
[`docs/aws.md`](docs/aws.md). One-time table setup: `npm run aws:table`.

See [`CLAUDE.md`](CLAUDE.md) for the full plan and [`docs/`](docs/) for the friction log,
product feedback, and submission notes.