# Dreamline

A voice-first dream journal for the **Build, Ship, Shape: Amazon Developer
Hackathon (Alexa+ track)**. You narrate your dreams each morning; Dreamline
stores them and finds longitudinal patterns — recurring symbols, themes, people,
places, and mood/stress correlations over weeks.

> **The hook:** *What if your dreams were data?*

Dreamline is a Python MCP server (FastMCP, Streamable HTTP, MCP spec 2025-11-25)
that fronts a SQLite dream store, an analysis engine (Bedrock Nova Lite +
Titan embeddings, with clustering and temporal statistics computed locally), and
a simulated Alexa voice flow.

---

## Two disclosures, up front

- **The Alexa layer is SIMULATED.** The real Alexa MCP Toolkit is partner-gated
  and US-only, and the hackathon rules permit a simulated host. `alexa_sim.py`
  prints the utterances a user would speak and the responses Alexa would give,
  and drives the real MCP tools behind the scenes. It is labeled SIMULATED in
  code, output, and here.
- **Seeded history is DEMO DATA.** `seed_demo.py` inserts ~3 weeks of
  **fictional** dreams, stored with `source="demo"` / `is_demo=1` and labeled
  "demo data" in code, output, and the insight narrative. It is never presented
  as a real person's dreams.

Dreamline runs **fully offline** with no AWS account: every Bedrock call
(Nova Lite extraction, Titan embeddings, narrative generation) degrades
gracefully to a deterministic local fallback, and every tool result prints
whether it used **Bedrock** or the **local fallback** (`ai_mode`).

---

## Zero → running server (under 10 minutes)

### 1. Use a modern Python (3.12+)

On this machine, bare `python3` is **3.9** and will fail `pip install`
(numpy/scipy need Python ≥ 3.12). Use an explicit interpreter.

```bash
# python3.13 and python3.14 are both present here; python3.12 is not installed.
python3.13 -m venv .venv        # (python3.14 also works; any python3.12+ works;
                                #  bare `python3` is 3.9 and WILL fail)
source .venv/bin/activate
```

### 2. Install dependencies

```bash
pip install -r requirements.txt
# optional, better clustering (never required — sklearn always provides HDBSCAN):
# pip install -r requirements-optional.txt
```

### 3. (Optional) enable Bedrock

Without this, Dreamline runs entirely in **local fallback mode**.

```bash
export AWS_REGION=ap-southeast-2   # APAC region for the Nova Lite inference profile
#   plus your standard AWS credentials (env vars / profile / instance role)
```

The Nova Lite model id (`apac.amazon.nova-lite-v1:0`) is an **APAC inference
profile**, so the client region must be an APAC region. A region/credential
mismatch simply lands you in local mode — that is expected and clearly logged.

### 4. Seed the labeled DEMO history (~3 weeks)

```bash
python seed_demo.py
```

### 5. Start the server (terminal 1)

```bash
python server.py                   # serves http://127.0.0.1:8000/mcp/
```

### 6. Prove every tool end-to-end (terminal 2)

```bash
python test_client.py              # calls all 4 tools, prints ai_mode, PASS/FAIL
```

> `test_client.py` uses a scratch DB. To share the demo history with the server,
> run both with the same `DREAMLINE_DB` (see **Environment variables** below).

### 7. Run the SIMULATED Alexa demo round trip (terminal 2)

```bash
python alexa_sim.py                # "log my dream" + "what patterns do you see?"
# python alexa_sim.py --seed       # also seed the DEMO history first
```

### 8. (Optional) extraction quality

```bash
python eval/evaluate.py            # precision/recall/F1 on the gold set
```

---

## The four MCP tools

| Tool | Behaviour |
|---|---|
| `log_dream(transcript, mood="", dream_date="")` | Save a narrated dream; auto-extract symbols/themes/emotions/people/places + a lucidity flag on write. |
| `get_dreams(limit=10, mood="", since="")` | List/filter past dreams, newest first. |
| `analyze_patterns()` | Aggregate analysis across all stored dreams + a grounded insight narrative. |
| `dream_stats()` | Counts, streaks, most common symbols, mood/valence trend. |

Every tool returns a uniform envelope: `{"ok": true, "ai_mode": "...", ...}` on
success, `{"ok": false, "error": "..."}` on failure.

## The analysis engine (not a CRUD wrapper)

1. **Feature extraction** — Nova Lite extracts structured features from a strict
   JSON prompt; a deterministic local extractor is the offline fallback.
2. **Embedding-based clustering** — dreams are embedded (Titan v2, or a local
   256-dim hashing embedder offline) and clustered (standalone HDBSCAN →
   `sklearn` HDBSCAN → KMeans) to surface emergent themes; cluster evolution
   (growing/shrinking/stable/emerging/fading) is tracked over time.
3. **Temporal statistics** — per-week symbol frequency, Jensen–Shannon
   divergence between early- and late-period theme mixes, a valence trend line,
   and a change-point scan.
4. **Grounded narrative** — the insight narrative may only state numbers the code
   actually computed. In Bedrock mode a verification pass discards any output
   that cites an unsupported number; in local mode a template fills the same
   fact sheet. Both cite only computed numbers.
5. **Evaluation** — a ~20-dream hand-labeled gold set with a precision/recall
   script (`eval/evaluate.py`).

---

## Environment variables

| Variable | Used by | Default | Purpose |
|---|---|---|---|
| `DREAMLINE_HOST` | `server.py` | `127.0.0.1` | Server bind host. |
| `DREAMLINE_PORT` | `server.py` | `8000` | Server port. |
| `DREAMLINE_DB` | all | `dreamline.db` | SQLite file path. |
| `DREAMLINE_URL` | `test_client.py`, `alexa_sim.py` | `http://127.0.0.1:8000/mcp/` | Server URL for clients. |
| `AWS_REGION` / `AWS_DEFAULT_REGION` | `bedrock.py` | *(unset → local mode)* | APAC region to enable Bedrock. |

To run the server and a client against the same database (e.g. so the client
sees the seeded demo history):

```bash
# terminal 1
DREAMLINE_DB=dreamline.db python server.py
# terminal 2
DREAMLINE_DB=dreamline.db python alexa_sim.py
```

---

## Troubleshooting

- **`pip install` fails with "No matching distribution found" for numpy/scipy.**
  You are on an old Python. Bare `python3` is 3.9 here; use `python3.13 -m venv
  .venv` (or `python3.14`; any python3.12+). `python3.12` is not installed on
  this machine.
- **`ai_mode` says `local` even though I have AWS credentials.** Set an APAC
  `AWS_REGION` (e.g. `ap-southeast-2`) — the Nova Lite id is an APAC inference
  profile, so a non-APAC region or missing model access falls back to local mode
  (by design). The server log explains why on first use.
- **A client prints "Could not reach the Dreamline server".** Start the server
  first (`python server.py`) and make sure `DREAMLINE_URL` points at
  `http://127.0.0.1:8000/mcp/`.

---

## Files

```
server.py          FastMCP Streamable HTTP server (the four tools)
extract.py         feature extraction (Nova Lite + local fallback) + vocabularies
analysis.py        clustering, temporal stats, grounded narrative
store.py           SQLite persistence
schema.sql         reference schema (mirrors store.py's DDL)
bedrock.py         Bedrock client factory + availability probe + mode flag
seed_demo.py       ~3 weeks of labeled DEMO DATA
alexa_sim.py       SIMULATED Alexa host (full demo round trip)
test_client.py     acceptance harness (exercises every tool over HTTP)
eval/              gold set + precision/recall script
requirements.txt   pinned deps (hdbscan intentionally NOT here)
```

## License

MIT. See [LICENSE](LICENSE).