Integrates with Amazon services that power the voice experience: Alexa+ as the patient-facing front door for describing symptoms and hearing care plans read back, and Amazon Bedrock for the ARIA intake model (GPT-OSS 120B) plus Titan Text Embeddings V2 embeddings used by the clinical memory.
Reads patient and care-plan data from the live Nirog doctor-portal Supabase project as an alternative data source, and stores clinical memory (past complaints, recurrence flags) in Supabase Postgres with pgvector, falling back gracefully when the store is unreachable.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@nirog-mcp I have a headache again, same as last month but worse"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Nirog for Alexa+ turns an Echo into the front door of a rural clinic.
A patient says what hurts, in their own words, in Hindi or English. The server records it, recalls what they said months ago however differently they worded it, asks the two or three questions a nurse would, and hands a doctor a real history. If it sounds dangerous, a fixed table of rules, not a model, tells them to call an ambulance and puts them at the top of the on-call doctor's queue.
The loop closes. The handover lands in the doctor's queue; the doctor files a prescription in the portal; that evening Alexa+ reads the new plan back and logs each dose, refusing a double one.
It is a self-hosted MCP server (Streamable HTTP) plus an Agent Skill, built for the Alexa+ track of the Build, Ship, Shape: Amazon Developer Hackathon. It also ships a web page that simulates the Alexa+ device against the real server, with a real agent (Amazon Nova on Bedrock) choosing the tools and Amazon Polly speaking, and the doctor's side of the same consultation.
Live: https://j2fkpyar4b.ap-south-1.awsapprunner.com/demo/ — press Play the 3-minute story (sound on).
Alexa+ (or the page's agent: Nova on Bedrock) ──MCP──▶ nirog-mcp ──▶ clinical memory (in-process | Postgres + pgvector)
│ ──▶ Amazon Bedrock (GPT-OSS 120B for questions, Titan Embed v2 for recall)
│ ──▶ clinic record (built-in demo set | Supabase, RLS on, append-only trust log)
Doctor portal ──/api/clinic──────────┘ one trust log, written by every tool callNeeds Node 22.9 or newer and pnpm. No database and no cloud account are required for the default setup.
pnpm install
pnpm build:demo # builds the web page into dist/demo
pnpm dev # server on http://127.0.0.1:3333Open http://127.0.0.1:3333/demo/. Scroll to Talk and click any of the five things to try, or use the bar at the bottom. Switch the bar to हिं to do the same in Hindi.
With no .env the server runs in open development mode: built-in demo patients, in-process memory seeded with a
demo history, an offline embedder, and ARIA's questions from Amazon Bedrock if your AWS credentials can reach it
(set ARIA_MODEL=off to use the built-in fallback questions instead).
pnpm test # 97 tests, offline, a few seconds
pnpm typecheckRelated MCP server: Daybreak
Tool | What it does |
| Records the complaint unedited, recalls earlier complaints that meant the same thing, resolves the body region (inheriting it from memory when the sentence names none), runs the recurrence rule and the triage rules, and returns ARIA's next question or a handover summary. A finished intake saves the handover for the doctor and queues the visit (urgent when the complaint keeps recurring). |
| Rules-based triage. On an emergency: speaks the 108 ambulance advice, queues a consult for the on-call doctor with the patient's words and the rule that fired, and puts the family contact on it. It answers even when the clinic record is down: the 108 advice never depends on the database. |
| Active medicines with times of day, pending tests, when the follow-up is due. |
| Records a dose and refuses a double dose: a second dose in the same part of the day, or any dose once today's are all on record. Days are counted in the clinic's time zone. |
| A caregiver's view of the last few days: complaint counts by body region, whether a pattern is forming, doses recorded against doses expected. Counts only, never the patient's words. |
| Health check. |
Every tool takes language (en or hi) and returns a spoken line in it. Every call is written to the trust
log with who, what and why.
Try them without the page:
npx @modelcontextprotocol/inspector # transport: Streamable HTTP, URL: http://127.0.0.1:3333/mcpThe split between code and model is the design, so it is worth stating exactly.
Decision | Made by | Why |
Which body region a complaint is about | A lexicon, then memory if the lexicon finds nothing | "The ache is back again" names no body part. Memory supplies one, but only from a close match; otherwise it stays unknown. |
Whether a pattern is worth a doctor's attention | Arithmetic: 3 visits, one region, 90 days | A doctor can check it on paper in ten seconds. |
Whether this is an emergency | 15 table-driven rules, negation-aware, condition-aware, in English and Hindi | Wheezing is an emergency for a patient with asthma and routine for anyone else. A model may raise a level, never lower one. |
Which earlier complaints mean the same thing | Embeddings (Titan Embed v2, or an offline lexical embedder) | This is the one job that needs meaning rather than words. |
Which tool the patient's words need | Amazon Nova Lite on Bedrock, reading the MCP tool descriptions (the page's simulated Alexa+) | That is Alexa+'s job. Code still passes the exact words, fixes the patient, speaks the tools' own lines, and escalates whatever the rules flag even if the model chose otherwise. |
What to ask next | GPT-OSS 120B on Amazon Bedrock, through a forced tool call | It conducts the conversation. It never writes the chart. Code holds it to at least two questions. |
What the doctor reads | Code: the patient's verbatim words, the region, the pattern, the triage level | An early version let the model write this, and it invented a symptom. Now nothing reaches the doctor that the patient did not say or a rule did not find. |
The lexicon, region inheritance, recurrence rule, embedders and SBAR handover are ported from the Nirog project (https://github.com/Shivang-creator/nirog, MIT). See CHANGELOG.md for what was built during the hackathon.
Copy .env.example to .env. Everything is optional.
Variable | Default | Effect |
| unset (open dev mode) | Turns on consent tokens for |
|
|
|
|
|
|
| Bedrock |
|
|
|
|
| unset | Enables Google sign-in on the doctor page. |
| demo values | One-time doctor verification. |
|
| The clock doses are counted on. |
| Nova Lite APAC, Nova 2 Lite, Nova Pro APAC | The agent's model chain, tried in order when one is throttled ( |
|
| The assistant's Polly voice (neural; Kajal speaks Indian English and Hindi). |
|
| Daily budget of audio |
| unset |
|
For Supabase: pnpm db:migrate creates the memory and clinic tables, seeds the demo clinic and history, and
resets today's queue when rerun. pnpm verify checks the connection, runs a live recall, reads the clinic data,
and confirms the public key is blocked and the trust log refuses edits. Before recording a demo,
pnpm db:reset -- --yes deletes what earlier runs created (consults, handovers, encounters, doses, complaints) and
reseeds; the trust log is append-only and keeps its history.
Households. Every /mcp request carries a consent token scoped to specific patients. A tool asked about any
other patient refuses, and does not reveal whether that patient exists.
pnpm consent-token pat_rahul,pat_aarav --sub yadav-household --days 30Doctors. The doctor page (/demo/#/doctor, or the "I'm a Doctor" button) has two gates: Google sign-in,
verified server-side, then a one-time check of the medical registration number and a clinic invite code against
an allowlist. The server issues a signed 30-day session, so later visits need only Google.
Patients. The patient page (/demo/#/patient, or "I'm a Patient") signs in with Google too, then binds the
account to one clinic record with a one-time link code the doctor sees on the chart. The session is a consent
token scoped to the patient and, for the account holder, their household: the same grant the Alexa+ tools and the
agent check, so the page can run the real intake for its own patient and nobody else. The page shows the patient's
own words, the summaries written for the doctor, the doctor's care plans, medicines and doses, and every trust-log
line about them. Details, endpoints, schema and limits: docs/PATIENT_PORTAL.md.
demo/ is a Vite + React page: glass panels over a Three.js particle sphere that reacts to the conversation.
The story. "Play the 3-minute story" resets the shared demo clinic and plays one evening from both chairs on the live page, with captions: it types into the real voice bar and presses the doctor's real buttons.
Voice bar. Follows the page. Microphone, typed input, suggestion chips, EN/हिं toggle. Each turn goes to
/api/agent, where Amazon Nova picks the MCP tools; the reply is the tools' own spoken lines, in Polly's voice.
The trace shows which model (or the patterns, or the guard) chose each tool. The microphone uses the browser's own
recognition where it has one; in Firefox, Brave and embedded browsers it records 16 kHz audio and /api/hear turns
it into words with Amazon Transcribe streaming (Indian English and Hindi).
Memory. A node graph of one patient's complaints, joined by the real cosine distances.
Case file. The recurrence verdict, the SBAR handover the doctor reads, a timeline in the patient's words.
Doctor portal. Queue sorted by triage, patients, chart, a consultation form that files an encounter, the
trust log, settings. With Supabase behind the server it reads and writes the live clinic through /api/clinic:
the handover Alexa+ saved opens beside the consult, the medicines still running are carried forward on the form,
and the filed plan is what Alexa+ reads next. Without a database it shows a built-in clinic and keeps changes in
the tab.
Talk. The conversation beside a trace of every tool call and what it returned.
Patient portal. Today, a labelled history timeline, medicines with "I took it" through the same dose tool, the conversation pinned to the signed-in patient, and "who looked at my record".
ARIA's face. On the patient's "Talk to Nirog" tab, the mobile app's 3D nurse speaks the replies, lip-synced to Polly's audio. The scene is the app's own, unmodified, in an iframe (docs/3D_MODEL_MIGRATION.md).
pnpm dev:demo # Vite on :5173 with hot reload, proxying to the server on :3333
pnpm agent:eval # how well the agent routes 26 labelled utterances (calls Bedrock)
pnpm demo:record # records the demo video from the live page: frames, Polly audio, title cards (needs ffmpeg)AWS App Runner in ap-south-1 (Mumbai), from an image in ECR.
bash deploy/iam.sh # once, as an account admin: two roles (ECR pull; Bedrock invoke and nothing else)
bash deploy/apprunner.sh # build, push, create or update the service, wait, print the HTTPS endpointThe second script refuses to deploy without NIROG_TOKEN_SECRET. It forwards the Supabase settings and
PUBLIC_DEMO from .env, so the deployed page works for a visitor with no Alexa+ account and no clinic login.
Stated plainly, because a clinical tool that overstates itself is worse than one that does less.
The agent's evaluation is small. 26 labelled utterances; a real Alexa+ deployment would need a far larger set, in more languages. A new AWS account's Bedrock quota is low; throttled turns fall back to other models, then to patterns, and the page says so.
Not tested against a real Alexa+ device. No Alexa+ sandbox was available. The server follows the MCP Streamable HTTP spec and is exercised by the MCP SDK client and MCP Inspector.
The triage rules are not clinically validated. They are a demonstration of the mechanism: a table a clinician can read and correct. They should be reviewed by one before any real use.
Recall across languages is unreliable. Measured on Titan Embed v2: the same back problem said in Hindi matches its English version (distance 0.781, threshold 0.85), but an unrelated Hindi complaint also falls inside the threshold (0.815), and romanised Hindi misses entirely (0.944). Recurrence is unaffected, because it counts by body region from the lexicon, and region inheritance uses a stricter cut-off (0.68). But a Hindi speaker may be reminded of an unrelated English complaint. Fixing it needs a labelled Hindi battery like the one the Nirog project built for English.
The model sometimes asks two things at once. The number of questions is enforced in code (two at least, three at most). Their wording is not: in our last run 1 of 38 model-written lines joined two questions with "and". One call in an earlier run of 44 returned no usable answer; the built-in question was spoken instead.
The family is not messaged. The household's contact is put on the consult for the doctor to call and the trust log records it. There is no SMS gateway, and the assistant never says the family has been told.
The public demo is open by design. With PUBLIC_DEMO=1 anyone can read and write the fictional demo clinic,
within a per-address rate limit. Real deployments leave it off and mint tokens per household.
The call room has no signalling server. It captures your camera and microphone and shows the waiting state; there is no second peer.
MIT
This server cannot be deployed
Telehealth support for Claude, Codex and Cursor: investigate cases and connect your tools.
Read and add to your own health record, kept in your own cloud, and get a health plan from it.
Consent-gated tools that turn user health notes into non-diagnostic appointment-prep materials.
Your own SLATE health data in your AI assistant: read, trend, plan and log. Never suggests a dose.