# memory-mcp-server

Unified semantic memory + time-series intelligence layer for OpenHome abilities.

## Architecture: memory tiers

```
Tier 1    — Semantic memory       entities, memories, relations, vectors
Tier 1.5  — Episodic memory       conversation sessions, turn-by-turn transcripts
Tier 1.75 — Working memory        short-lived task scratchpads, promote-on-close
Tier 2    — Time-series store     readings (numeric/categorical/composite), rollups, schedule
Tier 3    — Pattern engine        background task: promotes stable trends → Tier 1 memories
Tier 4    — Prospective memory    intentions: trigger_text → action_text, checked on each turn
Tier 5    — Spatial memory        last-known object locations with confidence decay
```

The pattern engine closes the loop: raw sensor data (Tier 2) automatically becomes
searchable, natural-language memory ("Brian's temperature preference is consistently 68°F")
that any ability can recall semantically (Tier 1).

## Files

| File              | Purpose                                              |
|-------------------|------------------------------------------------------|
| `server.py`       | MCP server (stdio transport, all 35 tools)           |
| `api.py`          | FastAPI HTTP wrapper + admin UI mount                |
| `admin.py`        | Admin UI router (served at `/admin`)                 |
| `voice_routes.py` | Speaker identity API (`/voices/*` — enroll, merge, update voiceprints) |
| `graph_routes.py` | Entity graph API (`/graph` SPA + `/api/graph` data endpoint) |
| `exporters/markdown.py` | Markdown two-way sync — export and import of Obsidian-compatible `.md` files |
| `reembed.py`      | Utility to re-embed all memories when swapping models|
| `templates/admin` | Jinja2 HTML templates for the admin UI               |
| `templates/graph` | vis.js entity graph SPA template                    |
| `integrations/`   | Standalone tools that connect external systems to memory-mcp via HTTP |

## Documentation

| Doc | Contents |
|---|---|
| `docs/overview.md` | What it is, motivation, integration patterns (OpenHome, HA, MQTT, IoT) |
| `docs/installation.md` | Requirements, step-by-step setup, first run, verification |
| `docs/quickstart.md` | First entity, memory, reading — common operations with curl examples |
| `docs/api-reference.md` | Full HTTP API — every endpoint, request/response shapes, examples |
| `docs/admin-ui.md` | Admin dashboard guide, pages, reading confidence, prune, security |
| `docs/ai-backend.md` | AI backend config, provider examples, model swap guide |
| `docs/pattern-engine.md` | How detectors work, all 5 detector types, how to add new ones |
| `docs/retention.md` | Retention policy config, what gets deleted, storage estimates |
| `docs/deployment.md` | systemd service, Docker Compose, reverse proxy, environment config |
| `docs/maintenance.md` | Keeping it healthy — backups, upgrades, model swaps, reembed.py walkthrough |
| `docs/testing.md` | Running tests, fixture design, what is and isn't covered |
| `docs/troubleshooting.md` | Common errors, what they mean, how to fix them |
| `integrations/README.md` | Integration index — MQTT bridge, HA state poller, OpenHome, Cloudflare |
| `integrations/background_example.py` | Background worker template — health data, environment sensors, weather |
| `integrations/ha_state_poller.py` | Pull-based HA state poller — polls HA REST API, pushes to memory-mcp |
| `integrations/homeassistant/README.md` | HA package setup — rest_commands, automations, scripts |
| `integrations/openhome/README.md` | OpenHome ability setup — background daemon + recall skill |
| `integrations/cloudflare/README.md` | Cloudflare Tunnel setup — safe internet exposure for cloud callers |

## Setup

```bash
# Install deps
pip install -r requirements.txt

# Pull embedding and LLM models (Ollama default)
ollama pull nomic-embed-text
ollama pull llama3.2

# Run MCP server (for OpenHome abilities)
python server.py

# Run HTTP API + admin UI (for HA webhooks, Node-RED, scripts)
python api.py          # listens on :8900
                       # admin UI at http://localhost:8900/admin/
```

## AI Backend

Uses the **OpenAI-compatible API** (`/v1/embeddings` + `/v1/chat/completions`).
Works with Ollama, OpenAI, LM Studio, Together AI, or any compatible provider.
Configure via environment variables — no code changes needed:

```bash
# Default (local Ollama)
export MEMORY_AI_BASE_URL=http://localhost:11434/v1
export MEMORY_EMBED_MODEL=nomic-embed-text   # 768-dim
export MEMORY_LLM_MODEL=llama3.2

# OpenAI
export MEMORY_AI_BASE_URL=https://api.openai.com/v1
export MEMORY_AI_API_KEY=sk-...
export MEMORY_EMBED_MODEL=text-embedding-3-small
export MEMORY_EMBED_DIM=1536
export MEMORY_LLM_MODEL=gpt-4o-mini
```

Split backends are supported — embed and LLM can run on different hosts:

```bash
# nomic-embed-text on a Raspberry Pi 4, LLM on a GPU machine
export MEMORY_AI_BASE_URL=http://pi4.local:11434/v1
export MEMORY_LLM_BASE_URL=http://gpu-host.local:11434/v1
```

See `docs/ai-backend.md` for full configuration guide, provider examples, and split backend setup.

## Source trust tiers

Every memory carries a trust tier that controls conflict resolution. When a new
fact is written, it can only supersede an existing contradicting memory if its
trust is **equal or higher**. Lower-trust sources cannot overwrite what you
explicitly told the system.

| Tier | Label | When to use |
|---|---|---|
| 5 | `user` | Direct user statements, manual entries via admin UI |
| 4 | `hardware` | Verified sensors, signed device data |
| 3 | `system` | Pattern engine promotions, LLM-extracted facts |
| 2 | `inferred` | Working memory promotions, low-confidence extractions |
| 1 | `external` | Third-party imports, unverified webhooks |

**Example:** a sensor (tier 4) recording "bedroom temperature is 68°F" will not
overwrite an explicit user statement (tier 5) "I keep my bedroom at 66°F at night."

Set via the `source_trust` parameter on `remember` / `POST /remember`.
Defaults to `MEMORY_TRUST_DEFAULT_REMEMBER` (env var, default: 5=user).

## Confidence decay

Memory confidence decays automatically over time so stale facts fade gracefully
instead of accumulating indefinitely. Decay runs every hour in the pattern engine.

**Formula:** `confidence = confidence × 2^(−days / halflife)`

| Category | Default half-life | Meaning |
|---|---|---|
| `preference`, `habit`, `routine` | 90 days | Stable — takes months to fade |
| `insight`, `general` | 90 days | Same default |
| `relationship` | 90 days | Same default |
| Location records | 24 hours | Unconfirmed location drops to 50% overnight |

Configure per-category overrides:
```bash
export MEMORY_DECAY_HALFLIFE_DAYS=90          # global default
export MEMORY_DECAY_CATEGORY_HALFLIFE='{"preference": 180, "insight": 30}'
export MEMORY_LOCATION_DECAY_HALFLIFE_HOURS=24
```

Use `GET /fading` (or `get_fading_memories`) to surface memories whose
confidence has dropped below a threshold — a prompt to confirm or update them.

## AI call timeout

```bash
export MEMORY_AI_TIMEOUT=30    # seconds; applies to both embed() and LLM calls
```

Increase if using a slow local model. LLM calls use `max(MEMORY_AI_TIMEOUT, 60)`
to guarantee at least 60 seconds for generation.

## Testing

```bash
pip install -r requirements.txt
python -m pytest                     # full suite (722 tests, no Ollama needed)
python -m pytest tests/test_tools.py # just tool tests
python -m pytest tests/test_spatial.py # just spatial/location tests
```

See `docs/testing.md` for fixture design and conventions.

## OpenHome SDK config

```json
{
  "mcpServers": {
    "memory": {
      "command": "python",
      "args": ["/path/to/memory-mcp/server.py"]
    }
  }
}
```

## Schema

```
TIER 1
  entities          id, name*, type, meta(JSON), created, updated
  memories          id, entity_id, fact, category, confidence, source, created, updated,
                    last_accessed, access_count, superseded_by
  relations         id, entity_a, entity_b, rel_type, meta(JSON), created,
                    valid_from, valid_until
  memory_vectors    rowid=memories.id, embedding FLOAT[768]   ← sqlite-vec

TIER 1.5
  sessions          id, entity_id, started_at, ended_at, summary, meta
  session_turns     id, session_id, role, content, ts

TIER 1.75
  working_memory_tasks   id, name, entity_id, status, ttl_ts, created, closed_at
  working_memory_slots   id, task_id, key, value(JSON), created, updated

TIER 2
  readings          id, entity_id, metric, unit, value_type, value_num,
                    value_cat, value_json, source, ts
                    (composite readings also decomposed into {metric}.{key} child rows)
  reading_rollups   id, entity_id, metric, bucket_type, bucket_ts,
                    count, avg_num, min_num, max_num, p10_num, p90_num, mode_cat
  rollup_watermarks entity_id, metric, last_ts   ← incremental build tracking
  schedule_events   id, entity_id, title, start_ts, end_ts, recurrence, meta, created

TIER 3
  promoted_patterns id, entity_id, metric, pattern_key, memory_id, detected

TIER 5
  locations         id, entity_id, container_id, container_name, confidence,
                    last_confirmed_ts, active, source, note, created
                    (active=1 → current location; active=0 → archived sighting)
```

### Entity types (open — add any string)
`person` | `house` | `room` | `device`

### Memory categories
`preference` | `habit` | `routine` | `relationship` | `insight` | `general`

### Value types for readings
| value_type    | field         | example                                   |
|---------------|---------------|-------------------------------------------|
| `numeric`     | value_num     | temperature=71.4, heart_rate=62           |
| `categorical` | value_cat     | mood="calm", presence="home"              |
| `composite`   | value_json    | `{"mood":"calm","confidence":0.91}`       |

## MCP Tools

### Tier 1 — Semantic memory
| Tool           | Description                                           |
|----------------|-------------------------------------------------------|
| `remember`     | Store a fact about any entity (embeds + indexes it)   |
| `recall`       | Semantic search — multi-factor: cosine × recency × confidence |
| `get_context`  | Relevance-filtered context snapshot (preferred for ability use) |
| `get_profile`  | Full profile: memories + relationships + readings     |
| `relate`       | Create directed relationship between entities         |
| `unrelate`     | Soft-delete a relationship (sets valid_until, preserves history) |
| `forget`       | Delete a memory or entire entity                      |
| `extract_and_remember` | LLM-powered fact extraction from conversation text |

### Tier 2 — Time-series
| Tool           | Description                                           |
|----------------|-------------------------------------------------------|
| `record`       | Ingest a reading (numeric/categorical/composite)      |
| `query_stream` | Query readings: raw or hour/day/week rollups          |
| `get_trends`   | Natural-language trend summary for a metric           |
| `schedule`     | Add a schedule event (one-off or recurring)           |

### Episodic memory
| Tool             | Description                                         |
|------------------|-----------------------------------------------------|
| `open_session`   | Open a conversation session for an entity           |
| `log_turn`       | Append a turn (user/assistant/system) to a session  |
| `close_session`  | Close a session with optional summary               |
| `get_session`    | Retrieve full session transcript                    |

### Working memory (Tier 1.75)
| Tool       | Description                                                          |
|------------|----------------------------------------------------------------------|
| `wm_open`  | Open a task-scoped scratchpad; optional TTL and entity association   |
| `wm_set`   | Write a key/value slot into an open task                             |
| `wm_get`   | Read one slot by key, or all slots with task metadata                |
| `wm_list`  | List tasks filtered by status (open/closed/expired/all) and entity   |
| `wm_close` | Close a task; optionally promote slots to long-term memory           |

### FTS keyword recall + session search
| Tool               | Description                                                      |
|--------------------|------------------------------------------------------------------|
| `recall` (mode=)   | Add `mode="keyword"` or `mode="hybrid"` for FTS5/BM25 recall — no embedding model needed |
| `search_sessions`  | Full-text search across episodic session turns (FTS5/BM25)       |

### Token-budget context assembly
| Tool                  | Description                                                   |
|-----------------------|---------------------------------------------------------------|
| `get_context_budget`  | Greedily fills a token budget with ranked memories + readings; `recall_mode="keyword"` for Pi/no-Ollama |

### Prospective / intention memory (Tier 4)
| Tool                 | Description                                                    |
|----------------------|----------------------------------------------------------------|
| `intend`             | Set a condition → action intention for an entity               |
| `check_intentions`   | Check if current text triggers any active intentions (FTS5)    |
| `dismiss_intention`  | Deactivate an intention                                        |
| `list_intentions`    | List active (or all) intentions for an entity                  |

### Spatial / location memory (Tier 5)
| Tool               | Description                                                       |
|--------------------|-------------------------------------------------------------------|
| `locate`           | Store or update where an object was last seen                     |
| `find`             | Return last known location with confidence + age ("where are my keys?") |
| `seen_at`          | Confirm object is still at its location; bumps confidence         |
| `location_history` | Full trail of past sightings in reverse-chronological order       |

### Cross-tier
| Tool           | Description                                           |
|----------------|-------------------------------------------------------|
| `cross_query`  | Semantic search across memories AND live readings     |

### Maintenance
| Tool                   | Description                                                    |
|------------------------|----------------------------------------------------------------|
| `prune`                | Delete raw readings older than `RETENTION_DAYS` (default 30d) |
| `get_fading_memories`  | Return memories whose confidence has fallen below a threshold, most faded first |

## HTTP API endpoints (api.py)

```
GET  /health                    liveness + row counts
GET  /entities                  list all entities
POST /remember                  store a memory
POST /recall                    semantic search (mode=vector|keyword|hybrid, recency_weight, min_confidence)
POST /get_context               relevance-filtered context snapshot
GET  /profile/{entity_name}     full profile
POST /relate                    create relationship
POST /forget                    delete memory or entity
POST /record                    ingest a reading
POST /record/bulk               ingest multiple readings at once
POST /query_stream              query time-series
POST /get_trends                trend summary
POST /schedule                  add schedule event
POST /cross_query               unified search
POST /prune                     delete readings older than RETENTION_DAYS
GET  /fading                    memories below a confidence threshold (most faded first)

POST /open_session              open a conversation session for an entity
POST /log_turn                  append a turn (user/assistant/system) to a session
POST /close_session             close a session with optional summary
GET  /get_session/{id}          retrieve full session transcript
POST /extract_and_remember      LLM-extract facts from text and store as memories

POST /wm/open                   open a working-memory task scope
POST /wm/set                    set a key/value slot in an open task
POST /wm/get                    get one slot or all slots from a task
GET  /wm/list                   list tasks (?status=open|closed|expired|all&entity_name=X)
GET  /wm/{task_id}              get all slots and metadata for a task
POST /wm/close                  close a task (promote=true bundles slots into long-term memory)

POST /locate                    store/update last-known location of an object
POST /find                      return last known location with confidence + age
POST /seen_at                   confirm object is still at a location; bumps confidence
GET  /location_history/{name}   full location trail for an object

POST /search_sessions           keyword search across session turn content (FTS5/BM25)
POST /get_context_budget        token-budget context snapshot (greedy fill, truncated flag)
POST /intend                    store a prospective intention (trigger_text → action_text)
POST /check_intentions          check if text triggers any active intentions (FTS5)
POST /dismiss_intention         deactivate an intention (soft-delete)
GET  /intentions                list intentions (?entity_name=X&active_only=true)

GET  /voices/unknown            list unenrolled provisional speaker entities
POST /voices/enroll             rename provisional entity to real person
POST /voices/merge              merge provisional entity into enrolled entity
POST /voices/update_print       update voiceprint embedding (running average)

GET  /graph                     vis.js entity relationship graph (SPA)
GET  /api/graph                 entity graph data { nodes, edges }

GET  /export/markdown           export all entities as Obsidian-compatible Markdown
GET  /export/markdown/{name}    export single entity as .md file download
POST /import/markdown           import entities from Markdown files (two-way sync)

GET  /admin/                    dashboard
GET  /admin/entities            entity list
GET  /admin/entity/{name}       entity detail
GET  /admin/readings            readings stream
POST /admin/prune               prune (HTMX-friendly HTML response)
```

## Usage examples

### Ability: build context before responding to Brian

```python
# Pull full profile (memories + latest readings + schedule)
profile = await mem.tool_get_profile("Brian")
# → inject as <memory>...</memory> in system prompt

# Or cross-query to pull what's relevant to the current question
context = await mem.tool_cross_query("how is Brian feeling today?")
```

### Home Assistant → record sensor readings via HTTP

```yaml
# configuration.yaml — rest_command
rest_command:
  push_temperature:
    url: http://localhost:8900/record
    method: POST
    content_type: application/json
    payload: >
      {"entity_name":"{{ room }}","metric":"temperature",
       "value":{{ temp }},"unit":"F","source":"ha","entity_type":"room"}

  push_presence:
    url: http://localhost:8900/record
    method: POST
    content_type: application/json
    payload: >
      {"entity_name":"{{ person }}","metric":"presence",
       "value":"{{ state }}","source":"ha"}

  push_mood:
    url: http://localhost:8900/record
    method: POST
    content_type: application/json
    payload: >
      {"entity_name":"{{ person }}","metric":"mood",
       "value":{"mood":"{{ mood }}","confidence":{{ conf }}},"source":"avatar_ability"}
```

### Avatar ability: store inferred mood state

```python
# After detecting mood from conversation
await mem.tool_record(
    entity_name="Brian",
    metric="mood",
    value={"mood": "focused", "confidence": 0.87},
    source="avatar_ability",
)
# The pattern engine will promote this to a memory like:
# "Brian's mood is predominantly 'focused' (72% of days)"
```

### Query last week of temperature with daily rollup

```python
result = await mem.tool_query_stream(
    entity_name="living_room",
    metric="temperature",
    granularity="day",
    start_ts=time.time() - 7 * 86400,
)
```

### Cross-entity semantic query

```python
result = await mem.tool_cross_query("who in the house prefers a cooler environment?")
# Returns: matching memories (explicit preferences) + live temperature readings scored by relevance
```

## Swapping embedding models

```bash
# 1. Set the new model and dimension via env vars
export MEMORY_EMBED_MODEL=mxbai-embed-large
export MEMORY_EMBED_DIM=1024

# 2. Pull the new model
ollama pull mxbai-embed-large   # 1024-dim — richer but slower

# 3. Re-embed all memories (non-destructive — only rebuilds memory_vectors)
python reembed.py --dry-run     # preview
python reembed.py               # run it
```

## Expanding the schema

- **New entity types**: pass any string — SQLite won't enforce the enum
- **New metric names**: pass any string to `record()` — fully dynamic
- **New memory categories**: same — add to enum in schema or free-text
- **New pattern detectors**: write `_detect_*(entity_name, metric, data) → list[tuple]`, call it via `_maybe_promote()` in `_promote_patterns()`. See `docs/pattern-engine.md`.
- **New rollup statistics**: add columns to `reading_rollups` and compute in `_build_rollups()`
- **Structured entity attributes**: use the `meta` JSON column on entities
  (e.g. `{"age": 35, "diet": "vegetarian", "wake_time": "06:30"}`)
- **Retention window**: change `RETENTION_DAYS` in `server.py`. See `docs/retention.md`.