# Echo Realtime Gateway

A Python asyncio service that turns a rooted Amazon Echo Dot into a full
speech to speech assistant. It bridges the device to Google Gemini Live for
native audio in and audio out, and it exposes the Echo hardware (LED ring,
volume, microphone, speaker, music) as an MCP surface that any local agent can
drive.

The Echo runs the open source EchoMuse firmware as its hardware runtime. This
repository is the gateway that the device dials into. The model API key stays on
the host that runs the gateway and is never copied to the device.

## Data path

```text
EchoMuse mic (16 kHz PCM)  ->  gateway  ->  Gemini Live
Gemini Live audio (24 kHz) ->  gateway  ->  EchoMuse speaker (48 kHz PCM)
```

The cloud connection sleeps while the device is idle. An on device wake word
starts a conversation, and native voice activity detection on the model side
handles turn taking and barge in.

## Features

- Native audio to audio bridge to Gemini Live, with full duplex barge in.
- Wake word start and idle auto close back to low power wake listening.
- MCP control surface for the Echo body: LED ring animations, volume, music,
  microphone, and speaker. A local agent consumes this over streamable HTTP.
- Swappable voice engines for the text based tools (echo_speak, echo_listen,
  echo_capture), local by default.
- Music playback on the device speaker, ducked under the assistant voice.
- Live control panel (optional) for watching device telemetry and editing the
  system prompt at runtime.
- Persona switching at runtime: hand the live call to a different assistant
  persona (its own system prompt and its own voice) without dropping the call.
- Optional worker integration: read only and action tools that let the voice
  assistant see and drive background agent workers, gated per worker.
- Optional conversation memory: each finished conversation is recorded, mixed to
  a single aligned track, and handed to a background worker that maintains one
  shared memory file. Every new session is seeded from that file so context
  survives restarts.

## Layout

```text
echo_gateway/
  main.py            process entry point and server wiring
  gateway.py         device link, Gemini bridge lifecycle, personas, recording
  gemini_bridge.py   Gemini Live client (audio, tools, text injection)
  echo_mcp.py        MCP server exposing the Echo body as tools
  orch_tools.py      optional worker control tools (gated)
  protocol.py        EchoMuse wire frames (mic, speaker, control)
  audio.py           resampling and speaker packetizing
  stt.py / tts.py    pluggable speech engines for the text tools
  telemetry.py       event hub for the control panel
  webui.py           optional live control panel
  assets/            earcon sounds (connect and disconnect)
tests/               unit tests for audio, protocol, MCP, and tools
```

## Requirements

- Python 3.12 or newer.
- ffmpeg on the PATH (used for music decode and recording mix).
- A rooted Echo Dot running EchoMuse firmware, reachable from the gateway host.
- A Google API key with access to a Gemini Live model, for gemini mode.

## Setup

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

cp .env.example .env
# edit .env: set GOOGLE_API_KEY and ECHO_MODE=gemini, adjust ports as needed
```

## Run

```bash
python -m echo_gateway.main
```

The gateway starts three things: the device facing server the Echo dials, the
MCP HTTP endpoint (default 127.0.0.1:8778), and, when enabled, the control
panel. Point the Echo at the device facing server, say the wake word, and talk.

## Configuration

All configuration is environment based. See `.env.example` for the full list.
The important ones:

- `ECHO_MODE` selects the bridge. Use `gemini` for the speech to speech
  assistant. `diagnostic` is a record and playback test that needs no key.
- `GOOGLE_API_KEY`, `GEMINI_LIVE_MODEL`, `GEMINI_LIVE_VOICE` configure the model.
- `ECHO_MCP_HTTP_HOST`, `ECHO_MCP_HTTP_PORT`, `ECHO_MCP_TOKEN` configure the MCP
  endpoint. Keep it bound to localhost unless you set a token.
- `ECHO_WAKE_ENABLED`, `ECHO_WAKE_PHRASE`, `ECHO_CONVERSATION_IDLE_SECONDS`
  control the wake and idle behavior.

Keys are read from the environment and are never written to any file in this
repository. The `.env.example` file ships with blank key fields on purpose.

## MCP surface

The gateway serves an MCP endpoint that routes to whichever Echo currently owns
the link. A local agent connects to it as an ordinary MCP client and gets tools
for the device body, for example `echo_set_led_solid`, `echo_animate_led`,
`echo_set_volume`, `echo_music_play`, `echo_music_stop`, `echo_get_state`, plus
the text based `echo_speak`, `echo_listen`, and `echo_capture`. The native audio
model does not use the audio tools, since it hears and speaks directly.

## Tests

```bash
python -m pytest tests -q
```

## Notes

- The worker integration tools and the conversation memory feature expect a
  companion background agent runtime on the same host. They are optional and the
  gateway runs fine without them.
- The panel `sample` music button expects a local `assets/sample.mp3`. That file
  is not shipped here; drop any mp3 at that path to enable the button, or ignore
  it.