# mnema

*(Greek μνῆμα — memory, memorial. Same root as Mnemosyne.)*

File-first personal memory layer for AI agents. Markdown is the source of
truth; SQLite is a disposable index. Conflicting memories are recorded, never
auto-deleted — resolution is a human decision, made when it is cheap to ask.

Born from a source-level audit of existing memory systems (memmy-agent,
Honcho, mem0). They converge on hybrid retrieval and LLM extraction — and
diverge on exactly the things mnema bets on: human-readable storage,
mandatory provenance, and honest conflict handling.

## Why another memory system

| Problem in existing systems | mnema's answer |
|---|---|
| Memory locked in a DB you can't grep, diff, or edit | Markdown files are canonical; the SQLite index rebuilds from them |
| Contradictions silently accumulate, or an LLM silently deletes the "old" fact | Conflicts become records with provenance; a human resolves; losers are downweighted, never deleted |
| Memories can't be traced back to their source | Every derived memory carries the session ref, message ids, and a redacted excerpt inline — auditable even after transcripts are purged |
| Everything gets stored, quality control deferred to retrieval | Extraction is an explicit, human-confirmed distill step with a write-time filter |

## Status: working, pre-release

Everything under Implemented is covered by the test suite (50 tests, offline
and deterministic) and has been exercised live against real Claude Code
sessions and the real Anthropic API.

## Quick start

```bash
npm install -g @raytien/mnema   # or: pnpm install && pnpm build (from source)
mnema --help

# manual memory
node dist/cli.js add --body "Always use pnpm, never npm" --stable

# hybrid search (first run downloads a ~118MB local embedding model;
# set MNEMA_NO_EMBED=1 for keyword-only, zero download)
node dist/cli.js search "package manager"

# extract memories from a Claude Code session (needs ANTHROPIC_API_KEY)
node dist/cli.js distill ~/.claude/projects/<proj>/<session>.jsonl --output run.json
# review run.json, then:
node dist/cli.js distill --apply run.json
```

MCP (read-only search from any Claude Code session):

```bash
claude mcp add mnema -- mnema-mcp
```

Env: `MNEMA_ROOT` (default `~/.mnema`), `MNEMA_NO_EMBED=1`,
`MNEMA_ENABLE_WRITE=1` (MCP write tools), `MNEMA_MODEL`, `ANTHROPIC_API_KEY`.

Recommended: `cd ~/.mnema && git init` — your memory history is just files.

## Implemented

**Storage — files first, crash-safe**
- Canonical markdown memories with Zod-validated frontmatter (ULID ids,
  versioned schema, discriminated source union)
- Write protocol: intent journal → cross-process lock → manifest generation →
  atomic file write (fsync + rename) → DB transaction. Fault-injection tests
  cover every crash point; recovery replays the journal without re-calling
  any LLM
- Repair-before-read across processes (durable dirty marker, not in-memory
  state); in-place index rebuild that never unlinks an open DB
- Manual edits detected by content hash: revision bumps, source wraps as
  `revised`, embeddings recompute — automatically, on the next index
- `op_key`/`op_hash` idempotency: same key replays, same key with a
  different payload errors (never silently dropped)

**Retrieval — hybrid, multilingual**
- FTS5 (contentless-delete, `remove_diacritics 2`) + local vector search
  (sqlite-vec, pinned multilingual MiniLM, q8) fused with RRF
- Shared tokenizer for index and query: `Intl.Segmenter` + Han bigrams —
  Chinese two-character terms actually hit (raw unicode61 scores zero);
  English/Spanish/Portuguese/French work as-is; `cafe` finds `café`
- Query hardening: user input never reaches FTS MATCH raw (`C++`,
  `alpha -beta`, emoji-only queries are all safe); input caps on every
  untrusted surface
- Time decay after fusion (stable memories exempt); superseded memories
  downweighted, derived from the resolution graph
- Cross-lingual retrieval via multilingual embeddings (verified live:
  English queries matching Chinese memories)

**Distill — explicit, audited capture**
- Claude Code JSONL parsed defensively (no public schema; bad lines counted,
  never a crash)
- Versioned redaction runs before anything reaches the LLM; secrets never
  survive into stored excerpts
- One extraction call per preview; write-time filter (preferences, decisions,
  constraints only — empty sessions honestly yield zero)
- `run.json` is immutable (tamper-detected by hash): accept/reject only;
  apply is LLM-free and idempotent
- Fabricated citations are dropped — provenance must be real

**Conflicts — record, never auto-delete**
- Batched LLM judging of semantically-near pairs; verdicts stored as
  deterministic relation files with the input hashes they were judged on
- Relations go `stale` when a member is edited, `orphaned` when deleted;
  stale verdicts stop affecting ranking
- `resolve keep:<id> | keep_both` re-verifies hashes under lock and rejects
  supersession cycles (A>B>C>A)
- Judge failures are recorded (`error`) and retryable — a network blip never
  becomes a permanently missed conflict
- Unresolved conflicts surface alongside search results

**Interfaces**
- CLI: `add / search / distill / resolve / check-conflicts / index / eval`
- MCP over stdio: read-only `search` by default; `add` and two-phase
  `resolve` (preview token required to commit) only behind an explicit flag
- Eval harness with a draft gold set; deterministic FTS-only baseline pinned
  in CI (recall@3 0.70, gate at 0.62)

**Security posture**
- `0700`/`0600` permissions, atomic temp-file writes, stdio-only MCP
- Honest residual risk: a poisoned conversation distilled into memory is
  persistent prompt injection; the mitigation is the human confirm step in
  distill, not a technical control

## Roadmap

Near-term (blocked on a confirmed gold set):
- [ ] Calibrate RRF k, decay half-life, and the conflict-candidate distance
      threshold (current values are literature defaults)
- [ ] Held-out test set and scheduled (non-CI) LLM quality evals: distill
      precision/recall, redaction precision, contradiction F1 with
      negative pairs

Planned:
- [ ] `mnema review` — batch conflict triage in the terminal
- [ ] Distill sources beyond Claude Code (Cursor, Codex session formats)
- [ ] Freshness re-verification (`source_status`) against still-existing
      transcripts
- [ ] Custom FTS tokenizer preserving symbol terms (`C++` vs `C` — currently
      a documented limitation; recall via the vector path only)
- [ ] npm publish + prebuilt binary matrix (macOS arm64, Linux x64)

Explicitly out of scope (v1 promises, not omissions):
- No agent runtime, no desktop app, no background daemon — one CLI, one MCP
  server, LLM calls only in explicit steps
- No auto-deletion of memories, ever
- No multi-user / workspace / auth — a personal, local tool
- Japanese/Korean text: detected and warned, not usefully indexed

## Design history

The full design history (comparison matrix, reviewed plan, implementation
specs, milestone tracking) is maintained privately.