Nova

Medication safety and care coordination for Alexa+

Nova catches medication mistakes before they happen and connects the signals a family would miss, for an older adult who lives alone and the people who care for her.

Demo video · Story page · Architecture · Submission notes

Contents

• Overview

• What Nova does

• Architecture

• Integrations

• Getting started

• Configuration

• MCP reference

• Testing

• Security, privacy and compliance

• Project structure

• Roadmap

• License

Related MCP server: Daybreak

Overview

More than 53 million Americans are unpaid family caregivers (AARP, Caregiving in the U.S., 2020), many of them far from the person they look after. The risky moments are predictable: a new dose after a doctor's visit, an old bottle still in the cabinet, a missed evening pill. The early warning signs are small and scattered: "I felt a bit woozy getting up", an aide's blood pressure reading, a quiet morning. Nobody sees them together until something goes wrong.

Nova brings them together on the devices the household already uses. It connects three places on Alexa+:

Nova is a self-hosted Model Context Protocol (MCP) server, the open standard Alexa+ uses for third-party experiences. This repository also includes a simulated Alexa+ (two Echo Shows and a Ring doorbell in the browser) whose agent is built for Amazon Nova on Amazon Bedrock, with a deterministic rules engine for the same tools, because Alexa+ developer preview access isn't generally available.

What Nova does

The main story: a prescription change, from paper to the pill bottle. After a cardiology visit, Dr. Patel lowers Rose's lisinopril from 20 mg to 10 mg.

1. Maya scans the new instructions. She photographs the after-visit summary on her Echo Show. Amazon Nova reads it, and Nova reconciles it with Rose's plan: change lisinopril 20 → 10 mg, because of dizziness when standing and low blood pressure.

2. She reviews and approves. Nothing changes until she does.

3. Rose hears the update. Her Echo explains it in plain words: "Rose, Dr. Patel changed your lisinopril from 20 mg once daily to 10 mg."

4. Rose shows the old bottle. She holds her old 20 mg bottle up to the camera: "Careful, Rose: this is the old 20 mg lisinopril. Your plan is now 10 mg, so please set this bottle aside."

5. Maya sees what happened and handles the follow-up. Her Echo tells her, and one tap starts the new 10 mg refill: "I've saved a simulated Lisinopril refill handoff. Please contact Elm Street Pharmacy to place the request."

One sequence shows vision, state that persists, human approval, two identities from two tokens, voice and screen, and MCP. In the live demo, Start the story walks through it step by step.

Safeguards around the story:

• Follow-ups have an owner. A caregiver can claim an alert, record what they did and resolve it, so the family doesn't double up. Everyone sees who is handling it.

• A doorbell is not an identity. A press near a scheduled visit asks Rose to confirm who is there; the aide's checklist and the family's arrival notice go out only after she does. Door-opening signals are labelled as door-sensor events.

• Every image card names its reader: Amazon Nova, or the sample label reader for the bundled demo photos. A patient-name mismatch on paperwork blocks approval.

• Refills are a handoff. Nova records the request and tells the family which pharmacy to contact; it doesn't place pharmacy orders itself.

It speaks first. A care engine runs on the household clock and turns the care plan into announcements on the right device.

It sees. Rose holds a bottle up to the Echo Show camera. Amazon Nova reads the label and Nova checks it against her plan: the right pill at the right time, a pill already taken, an outdated strength, or a medicine her doctor said to avoid ("Please don't take ibuprofen, Rose…"). Nova says take it only when it is sure: the exact product (not a combination or extended-release version), the strength read and matching the plan, and a confident read. Otherwise it asks her to check with the pharmacist, and it refuses outright when it can't read the label. Maya photographs an after-visit summary, and Nova proposes the plan changes (change, start, stop, continue, with the doctor's reasons). Nothing changes until she approves.

It connects the dots, with evidence. "How's Mom doing?" gets a one-sentence answer and a card that fits one screen:

Dizziness began after her lisinopril change. Home BP averaged 152/91 before the change and 110/66 since. Rose has mentioned dizziness four times since Oct 16. Grace saw her unsteady. Dr. Patel's notes say to call about dizziness.

Insights come from deterministic, tested rules, and each one carries its evidence. Amazon Nova writes the words. Nova never diagnoses.

It reads the front door, with Ring. Rose's Ring doorbell is a care signal, not a security camera:

• Ring Partner API. Nova reads the linked account's devices, whether the doorbell is online, its capabilities and its event history (doorbell presses and people at the door only). In the live demo, Front door ⋯ → Sync from Ring shows each call to api.amazonvision.com with its status, from a real Ring Developer Playground Doorbell Pro.

• Ring webhooks in real time. Webhook v1.1 events are verified with the X-Signature HMAC before they're parsed, and deduplicated by request ID.

• Expected visitors. A doorbell press during Grace's scheduled visit asks Rose "Who's at the door?". Once she confirms, Grace gets her visit checklist and the family hears she arrived.

• Signals a family would miss. A door opening at 2 AM sends an urgent alert; a morning with no pills and no movement at the door starts a wellness check; a doorbell going offline is noted on the timeline.

• Minimum necessary. Nova asks Ring only for the events it needs and stores no Ring email or address.

It stays respectful. Spoken replies are one or two sentences, with the details one "tell me more" away. Rose is asked before her family hears about a symptom, and she decides what the circle can see. Emergencies skip every question.

What we heard

We showed the demo to a friend and asked what's hardest and what worries them. Unedited:

"The hardest part is making sure they take the right medicine at the right time, especially when they have multiple medicines throughout the day. Sometimes they forget whether they've already taken a dose, and family members can't always be around to check."

"Yes, I think it could be helpful, especially for families who can't stay with their elderly parents all day. Automatic reminders and alerts for missed doses would provide peace of mind. My main concerns would be whether the camera can accurately identify the medicines, whether the system is easy for elderly people to use, and whether it protects their privacy. I'd also want to make sure it doesn't send false alerts or create confusion about whether a dose was actually taken."

One person's view, not a study; caregivers, nurses and pharmacists are next.

Architecture

flowchart LR
  subgraph Browser["Simulated Alexa+ (browser)"]
    R["Rose's Echo Show 8<br/>voice · camera"]
    D["Ring doorbell"]
    M["Maya's Echo Show 15"]
    I["Inspector: live MCP trace"]
  end
  subgraph Sim["simulator/ (FastAPI)"]
    A["Alexa+ agent<br/>Amazon Nova · Converse tool use"]
    H["MCP Apps host"]
    P["Announcement poller"]
    V["Amazon Polly voice"]
  end
  subgraph Nova["nova_server/ (MCP server)"]
    T["25+ tools · 8 resources · 2 prompts<br/>5 MCP Apps views"]
    E["Care engine<br/>(household clock)"]
    X["Insight engine"]
    S["Vision: pill check,<br/>reconciliation"]
    G["Ring: API client + webhook"]
    DB[("SQLite")]
  end
  R & M <-->|HTTP + SSE| Sim
  A <-->|"Streamable HTTP · OAuth 2.1 token per person"| T
  P <-->|resources/read| T
  D -->|signed webhook| G
  G <-->|Ring Partner API| RING["api.amazonvision.com"]
  T --> X & S --> DB
  E --> DB
  G --> E
  X -.->|language| BR["Amazon Bedrock<br/>Amazon Nova"]
  S -.->|images| BR
  A -.-> BR

Sequence diagrams for a voice turn, the escalation ladder, consent by elicitation, photo-to-plan reconciliation, account linking and the Ring integration are in docs/ARCHITECTURE.md.

Integrations

Alexa+ (Model Context Protocol)

• Streamable HTTP, negotiating MCP 2026-07-28 and serving 2025-11-25 clients.

• Account linking with OAuth 2.1: protected-resource metadata (RFC 9728), authorization-server metadata (RFC 8414), dynamic client registration (RFC 7591), PKCE, refresh-token rotation and revocation. Identity comes from the token, so the same tool answers differently for Rose and for Maya.

• MCP Apps (io.modelcontextprotocol/ui): five ui:// views (today, briefing, visit brief, pill check, reconciliation) render on the Echo Show screens and call tools back through the host. Camera-only tools are hidden from the model with _meta.ui.visibility: ["app"].

• Elicitation for consent, through SDK resolvers, and only when the client supports it.

• Structured output with an outputSchema on every tool, a short speech line for voice and more for "tell me more".

• Resources as a delivery queue: nova://announcements is how Nova speaks first (proactive notifications on Alexa+).

Amazon Nova on Amazon Bedrock

• Agent loop (simulator/agent.py): tool use through the Bedrock Converse API (Amazon Nova 2 Lite by default, with region-aware inference profiles and Nova Lite and Micro as fallbacks). When a tool returns a ready-to-speak line, the agent speaks it without a second model call.

• Vision (nova_server/vision.py): Amazon Nova reads pill-bottle labels and after-visit summaries. The reconciliation that follows is deterministic.

• Language (nova_server/insights.py): the caregiver briefing, structured check-in extraction, answers grounded only in the care notes, and visit briefs.

• Resilience: each turn can fall back to a local Ollama model, then a deterministic router, and Bedrock is re-checked in the background, so Amazon Nova takes over again without a restart.

Ring Partner API

• API client (nova_server/ring_api.py): GET /v1/users/me, /v1/devices, /v1/devices/{id}/status, /v1/devices/{id}/capabilities and /v1/history/devices/{id}/events?event_types=ding,motion.human, following the page[key] cursor. Nova asks only for doorbell presses and people at the door, and stores no Ring email or address.

• Webhooks (/integrations/ring/webhook): Webhook v1.1 events, with the X-Signature HMAC-SHA256 verified over the raw body before parsing, deduplication by meta.request_id, and an immediate 200.

• In the live demo, Front door → Sync from Ring shows each Ring API call with its status, labelled Live or Recorded.

Amazon Polly

Generative voices speak every reply and announcement in the simulator, and narrate the demo video.

Getting started

Requirements: Python 3.12+, uv, and Chrome (for the microphone and camera). AWS credentials are optional.

git clone https://github.com/Dakshhhhh-ops/nova && cd nova
uv sync
cp .env.example .env

uv run python run.py          # MCP server + simulator; opens the browser

Or run the two processes separately:

uv run python -m nova_server  # MCP server   http://127.0.0.1:8765/mcp
uv run python -m simulator    # simulator    http://127.0.0.1:8080  (story at /, live demo at /live)

A five-minute walkthrough

Open /live and press Start the story (top left). It runs the main story one step at a time:

1. Maya: Scan paperwork → after-visit summary. Nova reads the lisinopril change.

2. Approve. Rose's Echo explains the change.

3. Rose: Show Alexa → the old lisinopril bottle. Nova tells her to set it aside.

4. Maya: Order a 10 mg lisinopril refill, from the alert on her Echo. Nova records the refill handoff.

5. Fast-forward to 6 PM. A missed dose: a reminder, a follow-up, then Maya is told.

Then try the rest: Rose: Show Alexa → Ibuprofen (Nova stops her, citing her doctor's notes), Maya: "How's Mom doing?" then "Tell me more", and Front door → Grace arrives, then Rose says Yes, it's Grace after checking who is there.

Connect an MCP client

• MCP Inspector: npx @modelcontextprotocol/inspector, URL http://127.0.0.1:8765/mcp. Use its OAuth flow (Nova's account-linking page opens), or send the header Authorization: Bearer nova-demo-maya (also nova-demo-rose, nova-demo-leo and nova-demo-grace).

• stdio clients: uv run python -m nova_server --stdio --as rose.

Deploy

• Render: render.yaml is a one-click Blueprint. serve.py serves the story page, the live demo and the MCP server on one port.

• Containers: docker build -t nova ., then run with PORT and NOVA_PUBLIC_URL=https://your.host. Add AWS credentials or an instance role for Amazon Nova and Polly.

Configuration

All settings are environment variables (see .env.example).

MCP reference

Tools (25+)

Resources: nova://announcements, nova://circle, nova://timeline, and the five ui://nova/* views. Prompts: morning_checkin, caregiver_briefing.

Testing

uv run pytest -q

94 tests run in CI on every push. They cover:

• the care engine (the escalation ladder, snooze, check-ins and wellness checks);

• pill checks, including the safety rules for unreadable labels, low-confidence reads, missing strengths and look-alike products, and reconciliation;

• the dizziness insight and its evidence, BP limits, privacy, grounded answers and red flags;

• a real Streamable HTTP session with bearer auth and elicitation, and the full OAuth 2.1 account-linking flow;

• the Ring API client against recorded responses, and the Ring webhooks (signature, dedupe, event mapping);

• Bedrock request shapes against the botocore model, and Amazon Nova taking over again by itself after a fallback;

• the simulator's HTTP API end to end.

Security, privacy and compliance

• No diagnosis. Answers about medications and symptoms come from the care team's own notes, with sources. Plan changes need a person's approval.

• The older adult is in charge. Rose is asked before her family hears about a symptom, and only she can change what's shared.

• Emergencies are never gated. Falls, chest pain, breathing trouble and stroke signs alert everyone and tell her to call 911.

• Identity comes from the token, never from speech. Ring webhooks are verified before they're parsed.

• Minimum necessary. Nova requests only the Ring events it needs, keeps spoken replies general, and puts details on screen or behind "tell me more".

Running Nova with real health data makes it a HIPAA project. The operator (a home-care agency, health plan or pharmacy) runs Nova in its own AWS account under a Business Associate Addendum and uses HIPAA-eligible services only: Amazon Bedrock, Amazon Polly, Amazon ECS and Amazon RDS encrypted with AWS KMS, with TLS on every hop and an access log. Unlinking an account revokes Nova's tokens. The demo household, people, pharmacy and clinic are fictional, and this repository is not a HIPAA deployment as it stands.

Project structure

nova/
├── nova_server/        MCP server: tools, OAuth, care engine, insights, vision, Ring
│   └── views/          MCP Apps views (ui://nova/*)
├── simulator/          Simulated Alexa+: agent loop, MCP Apps host, devices
│   └── static/         Live demo (/live) and story page (site/)
├── samples/            Demo images and fixtures
├── skills/nova-care/   Agent Skill
├── scripts/            Demo-video recorder, UI checks
├── tests/              pytest suite
├── docs/               Architecture, design, demo script, submission notes
├── serve.py            Single-port server for deployment
└── run.py              Local launcher

Roadmap

• Production account linking with Login with Amazon or Amazon Cognito, and deployment on Amazon Bedrock AgentCore Runtime.

• Per-household Ring account linking and a registered webhook endpoint.

• Alexa proactive notifications for announcements.

• Amazon Nova Sonic for full-duplex voice, and ambient cards on Fire TV.

License

MIT. Built for the Amazon Developer Build, Ship, Shape hackathon. The open-source MCP Apps host used by the simulator is published separately as mcp-apps-lite-host.