Care Ladder · Alexa+ · Fire TV · MCP

Camera cue → multi-rung care ladder → Alexa+ agent (self-hosted MCP) → Fire TV caregiver audit (silhouette-only). Wellness ladder, not a medical device.

Product site: https://pamu512.github.io/care-ladder-amazon/

Hackathon map

Amazon Build, Ship, Shape (draft — no Final Submit).

MCP tools: start_or_resume_incident · check_in_prompt · advance_rung · resolve_incident · get_incident_status · notify_caretaker · request_call · caregiver_ack · caregiver_outcome · defer_escalation · tick_care_timers · how_is_household · confirm_schedule_pin

Proactive Events: optional awareness chime only. Default off (CARE_LADDER_PROACTIVE=0). Schema-locked; no rich buttons. Local sim only. See docs/proactive-events-honesty.md.

Related MCP server: EchoCare Kinship

Agentic proof (not thin MCP)

Session state is keyed by household + incident (household_id + incident_id). Every MCP tool return includes a session_snapshot (household_id, incident_id, rung, status, tools trail) so the agent carries one memory across turns — not disconnected FAQ calls.

The in-repo MCP client src/care_ladder/mcp_server/alexa_sim.py drives a real HTTP initialize → tools/call chain and prints a SESSION … rung=N status=… banner after each call:

1. start_or_resume_incident — open the incident

2. start_or_resume_incident again with the same ids — resume-same-incident (resumed: true)

3. check_in_prompt — fail-closed intent (clear_ok / needs_human / unclear)

4. advance_rung / notify_caretaker / resolve_incident — same incident id throughout

Soft “don’t worry” can stand the ladder down; “okay but hurt” will not. Both Path A soft-OK and needs-human print resume-same-incident and keep one incident id from start → check-in → resolve/notify.

Quickstart (≤60s to something on screen)

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# One story: start API → open Fire TV → sim drives the same store the TV polls
./scripts/amazon_demo_path.sh
# 1) http://127.0.0.1:8010/firetv/ stays open
# 2) alexa_sim --answer "don't worry"  → TV resolves (same incident)
# 3) second sim (needs-human)          → TV stays notify → Acknowledge

Manual split (same coupling):

CARE_LADDER_ALLOW_INSECURE_LOCAL=1 .venv/bin/uvicorn care_ladder.api.app:app --port 8010
# Fire TV:  http://127.0.0.1:8010/firetv/     (Demo console, bottom right)
# MCP:      POST http://127.0.0.1:8010/mcp    (Streamable HTTP)
.venv/bin/python -m care_ladder.mcp_server.alexa_sim --url http://127.0.0.1:8010 --answer "don't worry"

MCP tool calls write the incident into the AuditStore Fire TV polls (GET /incidents). A calm MCP agent active pill shows on the TV only while the sim is driving, then auto-clears. Keep /firetv/ visible — you should see the same incident change state within one sim run.

Amazon household: configs/amazon_demo_home.yaml (resident + primary/secondary contacts, stillness 4m). configs/demo_home.yaml is the legacy OpenCV plan, kept for before/after regression.

Judge remux (2:03, English VO): docs/demo/care-ladder-amazon-demo.mp4. Shot list in docs/demo-video-script-amazon.md. Remux notes: docs/demo/README.md.

Demo paths

Fire TV Demo console posts these fixtures to /demo/run. Or:

# Local scripts set CARE_LADDER_ALLOW_INSECURE_LOCAL=1. Hosted: add the bearer.
curl -s -X POST http://127.0.0.1:8010/demo/run \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer ${CARE_LADDER_API_TOKEN}" \
  -d '{"fixture":"alexa_path_a_soft_ok"}'

API auth & camera sources

/mcp and mutating routes (POST /demo/run, /demo/upload, /demo/camera/start|stop, incident ack, learning writes) require Authorization: Bearer $CARE_LADDER_API_TOKEN. If the token is unset those routes return 401 unless CARE_LADDER_ALLOW_INSECURE_LOCAL=1 (loopback demo / pytest only). See .env.example.

MCP Streamable HTTP has DNS-rebinding protection on (Host allowlist: localhost / testserver plus CARE_LADDER_MCP_HOSTS for an ALB hostname).

POST /demo/camera/start accepts a device index, or an rtsp/http(s) URL whose host is localhost / 127.0.0.1 / ::1 or listed in CARE_LADDER_CAMERA_HOSTS. Hostnames are resolved; cloud metadata IPs and arbitrary remote hosts are rejected.

Fire TV / console reads (GET /plan, GET /incidents) stay unauthenticated so the dashboard can poll. Fixture buttons, Acknowledge, and camera start/stop need the local opt-out or a bearer the page does not send — hosted token mode is for MCP / curl, not the in-page demo console.

Privacy & fail-closed

• Silhouette-only on Fire TV — no live video element, no raw frames.

• Frames that leave the device path go through blur_faces / to_silhouette before attach.

• Emergency dial is off and hard-locked in this demo build (hold-to-review explains; no call possible).

• Demo phones are reserved NANP fiction only: (555) 010-2276 (+12125550176).

• Footer copy: Wellness ladder — not a medical device.

Known limits: docs/failure-modes.md. Why this design: docs/research-brief.md.

Friction log

Honest tool/SDK notes (bonus): docs/friction-log.md.

Tests

.venv/bin/pytest tests/ -v

MCP handshake + care-flow tools, Fire TV HTML/API flow, response intent, alexa_sim client, and the shared ladder/API spine.

Fall-frame classifier (optional)

Kaggle download and training: docs/fall-cv-training.md. Weights land at models/fall_classifier.npz (gitignored). That npz is the only artifact (no ONNX). CueDetector.from_plan loads it when the file exists and CARE_LADDER_SAGEMAKER_ENDPOINT is unset. When the env var is set, the same weights are meant to run on SageMaker; the client sends blurred or silhouette features, not raw frames. Held-out table: docs/fall-cv-metrics.md. Endpoint setup and tear-down: infra/sagemaker/README.md.

AWS Builder

Alexa+ stays on self-hosted /mcp and alexa_sim. The Builder pieces are extra.

Architecture gallery refresh is a follow-up (Grok Bot). No Final Submit from this change.

Shared demo host (read-only from this repo)

Live HTTPS today: https://d2u7pls4da2poz.cloudfront.net/ — shared host from the opencv-care-ladder stack. CI on this repo is test-only (no auto-deploy) so OpenCV judges keep a stable CloudFront. Optional AWS replay: infra/README.md. Local run needs no AWS credentials: ./scripts/run_demo.sh.

Vision ONNX models and eval clips are not committed. If you want the optional vision-trigger proofs: ./scripts/download_models.sh and ./scripts/download_clips.sh. Alexa+ fixtures do not need ONNX.

Origins

Significant in-window update of OpenCV Care Ladder (pamu512/opencv-care-ladder). Vision cues are vendored here as the trigger; Amazon work is MCP + Fire TV + rungs. The OpenCV repo stays separate until after OpenCV judging. No Bee / Ring. No Bedrock foundation model.

License

MIT © 2026 Anoop Pamu. See LICENSE.