# nirog-mcp

Nirog for Alexa+ turns an Echo into the front door of a rural clinic.

A patient says what hurts, in their own words, in Hindi or English. The server records it, recalls what they
said months ago however differently they worded it, asks the two or three questions a nurse would, and hands a
doctor a real history. If it sounds dangerous, a fixed table of rules, not a model, tells them to call an
ambulance and puts them at the top of the on-call doctor's queue.

The loop closes. The handover lands in the doctor's queue; the doctor files a prescription in the portal; that
evening Alexa+ reads the new plan back and logs each dose, refusing a double one.

It is a self-hosted **MCP server** (Streamable HTTP) plus an **Agent Skill**, built for the Alexa+ track of the
Build, Ship, Shape: Amazon Developer Hackathon. It also ships a web page that simulates the Alexa+ device against
the real server, with a real agent (Amazon Nova on Bedrock) choosing the tools and Amazon Polly speaking, and the
doctor's side of the same consultation.

**Live:** https://j2fkpyar4b.ap-south-1.awsapprunner.com/demo/ — press **Play the 3-minute story** (sound on).

```
Alexa+ (or the page's agent: Nova on Bedrock) ──MCP──▶ nirog-mcp ──▶ clinical memory (in-process | Postgres + pgvector)
                                      │      ──▶ Amazon Bedrock (GPT-OSS 120B for questions, Titan Embed v2 for recall)
                                      │      ──▶ clinic record (built-in demo set | Supabase, RLS on, append-only trust log)
Doctor portal ──/api/clinic──────────┘          one trust log, written by every tool call
```

## Run it

Needs Node 22.9 or newer and pnpm. No database and no cloud account are required for the default setup.

```bash
pnpm install
pnpm build:demo     # builds the web page into dist/demo
pnpm dev            # server on http://127.0.0.1:3333
```

Open http://127.0.0.1:3333/demo/. Scroll to **Talk** and click any of the five things to try, or use the bar at
the bottom. Switch the bar to हिं to do the same in Hindi.

With no `.env` the server runs in open development mode: built-in demo patients, in-process memory seeded with a
demo history, an offline embedder, and ARIA's questions from Amazon Bedrock if your AWS credentials can reach it
(set `ARIA_MODEL=off` to use the built-in fallback questions instead).

```bash
pnpm test           # 97 tests, offline, a few seconds
pnpm typecheck
```

## The tools

| Tool | What it does |
|---|---|
| `start_intake` | Records the complaint unedited, recalls earlier complaints that meant the same thing, resolves the body region (inheriting it from memory when the sentence names none), runs the recurrence rule and the triage rules, and returns ARIA's next question or a handover summary. A finished intake saves the handover for the doctor and queues the visit (urgent when the complaint keeps recurring). |
| `report_red_flag` | Rules-based triage. On an emergency: speaks the 108 ambulance advice, queues a consult for the on-call doctor with the patient's words and the rule that fired, and puts the family contact on it. It answers even when the clinic record is down: the 108 advice never depends on the database. |
| `get_care_plan` | Active medicines with times of day, pending tests, when the follow-up is due. |
| `log_dose_taken` | Records a dose and refuses a double dose: a second dose in the same part of the day, or any dose once today's are all on record. Days are counted in the clinic's time zone. |
| `get_family_summary` | A caregiver's view of the last few days: complaint counts by body region, whether a pattern is forming, doses recorded against doses expected. Counts only, never the patient's words. |
| `ping` | Health check. |

Every tool takes `language` (`en` or `hi`) and returns a `spoken` line in it. Every call is written to the trust
log with who, what and why.

Try them without the page:

```bash
npx @modelcontextprotocol/inspector     # transport: Streamable HTTP, URL: http://127.0.0.1:3333/mcp
```

## What decides what

The split between code and model is the design, so it is worth stating exactly.

| Decision | Made by | Why |
|---|---|---|
| Which body region a complaint is about | A lexicon, then memory if the lexicon finds nothing | "The ache is back again" names no body part. Memory supplies one, but only from a close match; otherwise it stays unknown. |
| Whether a pattern is worth a doctor's attention | Arithmetic: 3 visits, one region, 90 days | A doctor can check it on paper in ten seconds. |
| Whether this is an emergency | 15 table-driven rules, negation-aware, condition-aware, in English and Hindi | Wheezing is an emergency for a patient with asthma and routine for anyone else. A model may raise a level, never lower one. |
| Which earlier complaints mean the same thing | Embeddings (Titan Embed v2, or an offline lexical embedder) | This is the one job that needs meaning rather than words. |
| Which tool the patient's words need | Amazon Nova Lite on Bedrock, reading the MCP tool descriptions (the page's simulated Alexa+) | That is Alexa+'s job. Code still passes the exact words, fixes the patient, speaks the tools' own lines, and escalates whatever the rules flag even if the model chose otherwise. `pnpm agent:eval`: 26/26. |
| What to ask next | GPT-OSS 120B on Amazon Bedrock, through a forced tool call | It conducts the conversation. It never writes the chart. Code holds it to at least two questions. |
| What the doctor reads | Code: the patient's verbatim words, the region, the pattern, the triage level | An early version let the model write this, and it invented a symptom. Now nothing reaches the doctor that the patient did not say or a rule did not find. |

The lexicon, region inheritance, recurrence rule, embedders and SBAR handover are ported from the Nirog project
(https://github.com/Shivang-creator/nirog, MIT). See [CHANGELOG.md](CHANGELOG.md) for what was built during the
hackathon.

## Configuration

Copy `.env.example` to `.env`. Everything is optional.

| Variable | Default | Effect |
|---|---|---|
| `NIROG_TOKEN_SECRET` | unset (open dev mode) | Turns on consent tokens for `/mcp` and doctor sessions for `/api/clinic`. Required for any deployment. |
| `MEMORY_STORE` | `memory` | `pg` stores complaints in Postgres at `MEMORY_DATABASE_URL` (`MEMORY_PG_FLAVOUR`: `pgvector` for Supabase, `cockroach` for CockroachDB). |
| `EMBEDDER` | `offline` | `bedrock` uses Titan Text Embeddings V2 in `AWS_REGION`. |
| `ARIA_MODEL` | Bedrock | `off` uses built-in questions. `BEDROCK_CHAT_MODEL` picks the model. |
| `NIROG_DATA_SOURCE` | `mock` | `supabase` uses the clinic schema in `supabase/migrations/` (`SUPABASE_URL`, `SUPABASE_SERVICE_KEY`). |
| `GOOGLE_CLIENT_ID` | unset | Enables Google sign-in on the doctor page. |
| `CLINIC_INVITE_CODE`, `DOCTOR_ALLOWLIST` | demo values | One-time doctor verification. |
| `CLINIC_UTC_OFFSET_MINUTES` | `330` (IST) | The clock doses are counted on. |
| `BEDROCK_AGENT_MODELS` | Nova Lite APAC, Nova 2 Lite, Nova Pro APAC | The agent's model chain, tried in order when one is throttled (`BEDROCK_AGENT_MODEL` pins one). |
| `POLLY_VOICE` | `Kajal` | The assistant's Polly voice (neural; Kajal speaks Indian English and Hindi). |
| `TRANSCRIBE_DAILY_SECONDS` | `3600` | Daily budget of audio `/api/hear` sends to Amazon Transcribe, per instance. |
| `PUBLIC_DEMO` | unset | `1` on the hackathon deployment only: visitors get a 6-hour consent token for the fictional demo household and a demo doctor session, and `/mcp` is rate-limited per address. Never with real patients. |

For Supabase: `pnpm db:migrate` creates the memory and clinic tables, seeds the demo clinic and history, and
resets today's queue when rerun. `pnpm verify` checks the connection, runs a live recall, reads the clinic data,
and confirms the public key is blocked and the trust log refuses edits. Before recording a demo,
`pnpm db:reset -- --yes` deletes what earlier runs created (consults, handovers, encounters, doses, complaints) and
reseeds; the trust log is append-only and keeps its history.

## Consent and sign-in

**Households.** Every `/mcp` request carries a consent token scoped to specific patients. A tool asked about any
other patient refuses, and does not reveal whether that patient exists.

```bash
pnpm consent-token pat_rahul,pat_aarav --sub yadav-household --days 30
```

**Doctors.** The doctor page (`/demo/#/doctor`, or the "I'm a Doctor" button) has two gates: Google sign-in,
verified server-side, then a one-time check of the medical registration number and a clinic invite code against
an allowlist. The server issues a signed 30-day session, so later visits need only Google.

**Patients.** The patient page (`/demo/#/patient`, or "I'm a Patient") signs in with Google too, then binds the
account to one clinic record with a one-time link code the doctor sees on the chart. The session is a consent
token scoped to the patient and, for the account holder, their household: the same grant the Alexa+ tools and the
agent check, so the page can run the real intake for its own patient and nobody else. The page shows the patient's
own words, the summaries written for the doctor, the doctor's care plans, medicines and doses, and every trust-log
line about them. Details, endpoints, schema and limits: [docs/PATIENT_PORTAL.md](docs/PATIENT_PORTAL.md).

## The web page

`demo/` is a Vite + React page: glass panels over a Three.js particle sphere that reacts to the conversation.

- **The story.** "Play the 3-minute story" resets the shared demo clinic and plays one evening from both chairs on
  the live page, with captions: it types into the real voice bar and presses the doctor's real buttons.
- **Voice bar.** Follows the page. Microphone, typed input, suggestion chips, EN/हिं toggle. Each turn goes to
  `/api/agent`, where Amazon Nova picks the MCP tools; the reply is the tools' own spoken lines, in Polly's voice.
  The trace shows which model (or the patterns, or the guard) chose each tool. The microphone uses the browser's own
  recognition where it has one; in Firefox, Brave and embedded browsers it records 16 kHz audio and `/api/hear` turns
  it into words with Amazon Transcribe streaming (Indian English and Hindi).
- **Memory.** A node graph of one patient's complaints, joined by the real cosine distances.
- **Case file.** The recurrence verdict, the SBAR handover the doctor reads, a timeline in the patient's words.
- **Doctor portal.** Queue sorted by triage, patients, chart, a consultation form that files an encounter, the
  trust log, settings. With Supabase behind the server it reads and writes the live clinic through `/api/clinic`:
  the handover Alexa+ saved opens beside the consult, the medicines still running are carried forward on the form,
  and the filed plan is what Alexa+ reads next. Without a database it shows a built-in clinic and keeps changes in
  the tab.
- **Talk.** The conversation beside a trace of every tool call and what it returned.
- **Patient portal.** Today, a labelled history timeline, medicines with "I took it" through the same dose tool,
  the conversation pinned to the signed-in patient, and "who looked at my record".
- **ARIA's face.** On the patient's "Talk to Nirog" tab, the mobile app's 3D nurse speaks the replies, lip-synced
  to Polly's audio. The scene is the app's own, unmodified, in an iframe ([docs/3D_MODEL_MIGRATION.md](docs/3D_MODEL_MIGRATION.md)).

```bash
pnpm dev:demo       # Vite on :5173 with hot reload, proxying to the server on :3333
pnpm agent:eval     # how well the agent routes 26 labelled utterances (calls Bedrock)
pnpm demo:record    # records the demo video from the live page: frames, Polly audio, title cards (needs ffmpeg)
```

## Deploy

AWS App Runner in `ap-south-1` (Mumbai), from an image in ECR.

```bash
bash deploy/iam.sh         # once, as an account admin: two roles (ECR pull; Bedrock invoke and nothing else)
bash deploy/apprunner.sh   # build, push, create or update the service, wait, print the HTTPS endpoint
```

The second script refuses to deploy without `NIROG_TOKEN_SECRET`. It forwards the Supabase settings and
`PUBLIC_DEMO` from `.env`, so the deployed page works for a visitor with no Alexa+ account and no clinic login.

## Known limits

Stated plainly, because a clinical tool that overstates itself is worse than one that does less.

- **The agent's evaluation is small.** 26 labelled utterances; a real Alexa+ deployment would need a far larger
  set, in more languages. A new AWS account's Bedrock quota is low; throttled turns fall back to other models, then
  to patterns, and the page says so.
- **Not tested against a real Alexa+ device.** No Alexa+ sandbox was available. The server follows the MCP
  Streamable HTTP spec and is exercised by the MCP SDK client and MCP Inspector.
- **The triage rules are not clinically validated.** They are a demonstration of the mechanism: a table a
  clinician can read and correct. They should be reviewed by one before any real use.
- **Recall across languages is unreliable.** Measured on Titan Embed v2: the same back problem said in Hindi
  matches its English version (distance 0.781, threshold 0.85), but an unrelated Hindi complaint also falls
  inside the threshold (0.815), and romanised Hindi misses entirely (0.944). Recurrence is unaffected, because
  it counts by body region from the lexicon, and region inheritance uses a stricter cut-off (0.68). But a
  Hindi speaker may be reminded of an unrelated English complaint. Fixing it needs a labelled Hindi battery
  like the one the Nirog project built for English.
- **The model sometimes asks two things at once.** The number of questions is enforced in code (two at least,
  three at most). Their wording is not: in our last run 1 of 38 model-written lines joined two questions with
  "and". One call in an earlier run of 44 returned no usable answer; the built-in question was spoken instead.
- **The family is not messaged.** The household's contact is put on the consult for the doctor to call and
  the trust log records it. There is no SMS gateway, and the assistant never says the family has been told.
- **The public demo is open by design.** With `PUBLIC_DEMO=1` anyone can read and write the fictional demo clinic,
  within a per-address rate limit. Real deployments leave it off and mint tokens per household.
- **The call room has no signalling server.** It captures your camera and microphone and shows the waiting
  state; there is no second peer.

## Licence

MIT