InnerOS Ambient Guardian

Alexa becomes the voice of a local-first AI guardian that explains home incidents, prepares bounded actions, requires separate human approval, and returns verified evidence.

Built for Build, Ship, Shape: Amazon Developer Hackathon 2026.

• Primary track: Alexa+

• Mini challenges: AWS Builder, Open Source

• Ring track: the private Ring app and its official Amazon Vision Playground flow are OFFICIAL_SANDBOX_VERIFIED; the official sandbox returned a device, status, capabilities, and a WHEP session. Physical-device/provider-live evidence remains unclaimed, and local demo events stay visibly SIMULATED.

Why this exists

Smart-home systems produce many alerts but still make a person answer the hard questions manually: What happened? Is it important? What should happen next? Did the action actually work?

Ambient Guardian gives Alexa a safe orchestration surface through an official MCP server and a signed Custom Skill gateway. It correlates property context, reasons locally, prepares only allowlisted actions, waits for a separate human approval, executes through an adapter, verifies observed state, and records evidence. Direct Alexa+ MCP remains explicitly CODE_READY / NOT_PROVEN until the account and device entitlement are demonstrated.

The core invariant is deliberately strict:

No human approval, no consequential physical action. Low-risk reversible ambience commands are separately allowlisted. No verification, no success claim.

The public hackathon build controls only a simulator. Private customer/device configuration is not copied into this repository.

Related MCP server: ProvenDone

What is functional

• Official MCP Python SDK v2 server at /mcp

• Streamable HTTP transport, modern MCP protocol with backward compatibility for the hackathon-required 2025-11-25 generation

• MCP tools for status, events, time-bounded incident reconstruction, local reasoning, action preparation, evidence, and integration diagnostics

• No approve_action MCP tool. A model can prepare an action but cannot approve its own request

• Alexa+ web simulation with typed input, browser speech recognition, and spoken responses

• Signed physical-Alexa Custom Skill gateway with Amazon certificate, timestamp, Skill ID, Account Linking, Voice ID/PIN, and a shared-secret loopback trust boundary

• Live Evidence Console for utterance, intent, consulted systems, Incident Receipt, prepared action, approval, PBX result, evidence digest, and REAL/SIMULATED/CODE_READY labels

• Ring-compatible normalized demo ingress for safe public testing, plus a separate official Ring/Amazon Vision adapter with fail-closed signed webhook verification. Demo events remain explicitly SIMULATED unless a real provider session/signature is verified.

• Optional Home Assistant + Alexa Devices bridge for real Echo/Fire endpoints, read-only alarm context, owner-authorized speech, and allowlisted reversible lighting; direct Alexa+ MCP Toolkit linking remains separate and unclaimed

• InnerOS DMX / Art-Net bridge to the existing loopback-only lighting engine for allowlisted colors/scenes Alexa does not natively understand

• Real Home Assistant timeline for alarm zones, lights, solar/battery transitions, and network availability, kept separate from truth-labeled Ring demo events

• Two-phase UniFi restart for explicitly allowlisted devices: Alexa prepares; InnerOS sends an authenticated WhatsApp approval; only the owner confirmation executes and checks state/uptime evidence

• Local Qwen/vLLM reasoning using an OpenAI-compatible endpoint

• Deterministic local fallback if the LLM is unavailable

• AWS Strands Agents SDK as a real read-only orchestration/synthesis layer against the local OpenAI-compatible Qwen endpoint

• One-time expiring approval tokens, replay protection, and concurrent-consumption protection

• Post-action verification evidence

• Persistent Home Flight Recorder / Incident Receipts backed by SQLite, with immutable incident IDs and SHA-256 digests recoverable across sessions

• Action evidence digests linked back to the latest incident receipt

• VoiceOps/PBX callback preparation linked to the exact Incident Receipt that existed at PREPARE time; VoiceOps alone owns the private extension and SIP/PBX credentials

• Docker packaging and CI that boots the actual server and connects with a real MCP HTTP client

Architecture

Physical Echo Custom Skill / Judge Mode / entitlement-gated Alexa+ MCP
        |
        v
Official MCP Python SDK v2 / Streamable HTTP
        |
        +--> read-only context tools
        |       |
        |       +--> AWS Strands Agent --> local OpenAI-compatible Qwen/vLLM
        |       +--> deterministic local fallback
        |
        +--> prepare_action (never executes)
                    |
                    v
       INNEROS OWNER WHATSAPP APPROVAL
              (not an MCP/web tool)
                    |
                    v
              adapter.execute()
                    |
                    v
                 verify()
                    |
                    v
                 evidence

The Strands agent is intentionally created without physical-action tools. It can synthesize context and recommendations; deterministic application code owns action parsing, authorization, execution, and verification.

More detail: docs/ARCHITECTURE.md and docs/SECURITY.md.

Quick start

python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
PYTHONPATH=src python -m ambient_guardian.official_server

Open:

• UI: http://127.0.0.1:8787/

• Health: http://127.0.0.1:8787/health

• MCP: http://127.0.0.1:8787/mcp

Run all tests:

PYTHONPATH=src pytest -q

Run an actual Streamable HTTP MCP client against the running server:

PYTHONPATH=src python scripts/http_mcp_smoke.py

Local-first Qwen / vLLM

Point the app at any OpenAI-compatible local endpoint:

export INNEROS_LOCAL_LLM_URL=http://127.0.0.1:8000
export INNEROS_LOCAL_LLM_MODEL=QuantTrio/Qwen3-Coder-30B-A3B-Instruct-AWQ

Sensitive local addresses are runtime configuration, never committed to the public repository.

AWS Strands Builder path

Strands is installed as part of the standard project environment so the AWS Builder integration is reproducible. Enable it explicitly:

export AWS_STRANDS_ENABLED=1
export AMBIENT_GUARDIAN_STRANDS_PROVIDER=local-openai
export INNEROS_LOCAL_LLM_URL=http://127.0.0.1:8000
PYTHONPATH=src python scripts/strands_local_smoke.py

The integration uses strands.Agent with strands.models.openai.OpenAIModel, pointed at the local vLLM OpenAI-compatible endpoint. Bedrock is an optional provider, not a dependency of the safety path.

Demo flow

The strongest Judge Mode path is a three-part story rather than a feature tour:

1. Home status: ask Alexa, is everything okay at home? to prove read-only context.

2. What happened at 3 AM?: the demo injects truth-labeled Ring-compatible events at 03:04, 03:07 and 03:11, then reconstructs the incident through the real normalized event pipeline and the read-only incident_summary MCP tool.

3. Prepare lock: ask Alexa, prepare to lock the front door. The system returns an expiring proposal with executed=false. Only the separate human approval surface may execute the bounded simulator action and emit verification evidence.

4. PBX escalation: ask Alexa, call me through my PBX and give me the incident report for the private softphone alias, or Alexa, ask Atlas Control to call my cell phone using Grandstream for the single owner-mobile alias. Both are prepare-only in Alexa; InnerOS sends WhatsApp approval, VoiceOps rejects supplied numbers, and a confirmed request records sanitized verification evidence.

5. Governed UniFi restart: ask Alexa to restart an allowlisted UniFi device. Alexa only prepares it; InnerOS sends Confirm/Cancel to the enrolled owner's WhatsApp. Approval is durable, expires after a bounded interval, executes at most once, and sends verified completion back to WhatsApp.

The demo always distinguishes the SIMULATED Ring edge from the REAL MCP/runtime, temporal correlation, policy and evidence pipeline. The 3 AM scenario also writes a persistent Incident Receipt so the same incident can be recovered in a later session.

MCP tools

There is deliberately no MCP execution/approval tool.

Alexa, PBX, and Ring readiness

The project reports each integration separately so a code-complete path is never presented as live evidence:

• Physical Echo via Custom Skill: code-complete signed gateway at /alexa/skill, with owner Account Linking and authenticated gateway-to-backend identity forwarding. It remains CODE_READY until a physical request/response is captured.

• Direct Alexa+ MCP Toolkit: entitlement is NOT_PROVEN; it is not claimed as LIVE.

• VoiceOps/PBX: the app sends only a stable server-side alias, never a number, extension, or SIP credential. After WhatsApp owner approval it calls the private VoiceOps contract and verifies the dispatch receipt. voiceops.owner.mobile is limited to one preconfigured route-verified number; arbitrary and autonomous PSTN stay disabled. Simulator mode is explicitly SIMULATED.

• Ring: the official adapter targets https://api.amazonvision.com/v1, verifies signed webhook ingress fail-closed, and normalizes JSON:API events. The official Ring Playground device/status/capabilities/WHEP path is verified. A physical device and signed live webhook remain unclaimed; local demo ingress is always labelled SIMULATED.

Alexa speech from Ambient Guardian is intentionally not an MCP tool. It requires a separate owner-authorized HTTP route, an explicit enable flag, and an allowlisted Home Assistant notify entity.

Docker

docker build -t inneros-ambient-guardian .
docker run --rm -p 8080:8080 inneros-ambient-guardian

See docs/DEPLOYMENT.md for production host/origin settings.

Environment variables

Testing and evidence

CI performs all of the following from a clean environment:

1. installs declared dependencies,

2. compiles source/tests/scripts,

3. runs the full pytest suite,

4. boots the official MCP + web server,

5. connects to /mcp with the official MCP client over real HTTP,

6. verifies tool discovery and calls,

7. builds the Docker image.

The security regression suite includes unlock/negation parsing, token expiration, replay, concurrent approval consumption, durable action recovery, and WhatsApp approve/reject behavior.

Final-sprint reproducible result on 2026-10-13: 112 tests passed, compileall passed, and scripts/runtime_smoke.py connected to the running Streamable HTTP server with the official MCP client. Tool discovery confirmed that MCP exposes preparation but no approval, execution, or speaker tool.

Hackathon evidence

• Amazon integration notes: docs/AMAZON_INTEGRATIONS.md

• Friction log: docs/FRICTION_LOG.md

• Demo outline: docs/DEMO_SCRIPT.md

• Devpost draft: devpost-submission.md

Links

• Devpost: https://devpost.com/software/inneros-ambient-guardian

• Repository: https://github.com/Rafa-Innerchispa/inneros-ambient-guardian-amazon-2026

License

MIT. See LICENSE.

Hackathon Judge Mode

Judge Mode preserves three deterministic scenarios and adds a Live Evidence Console. The final physical video path uses the Custom Skill when physical verification is available; the browser remains the reproducible fallback and never impersonates an Echo.

Open the web UI and use the three Judge Mode scenarios:

1. Home status — read-only property context.

2. What happened at 3 AM? — truth-labeled Ring-compatible demo events -> real temporal incident reconstruction.

3. Prepare lock — bounded proposal with executed=false until a separate human approval step.

Truth boundary shown in the UI:

• REAL: MCP Streamable HTTP runtime and Guardian policy/state.

• CODE_READY: signed Alexa Custom Skill gateway, until physical proof is captured.

• CODE_READY / NOT_PROVEN: direct Alexa+ MCP entitlement.

• OFFICIAL_SANDBOX_VERIFIED: the Amazon Ring Playground returned a sandbox device, online status, AVC 1080p capabilities, and a successful WHEP motion session.

• SIMULATED: the event injected into Atlas remains a truth-labelled Ring-compatible fixture. A real owner account, physical device, and signed webhook are still pending.

• SAFE: MCP/model cannot approve or execute its own physical action.

See docs/JUDGE_DEMO.md and docs/DEMO_SCRIPT.md for the reproducible judge flow and recording script.