retained

An MCP memory server whose writes are provable.

A memory server answers ok and you move on. That answer is a promise about bytes, and almost nobody tests it. retained makes the promise checkable: every write is appended, fsynced, read back from the file and hash-chained to its predecessor before it is acknowledged; every read says how much it did not return; and the search index can be reconciled against the log it came from, in both directions.

$ python retained.py --selftest
retained selftest: 14 ok, 0 failed

Three invariants

The store is the truth: an append-only JSONL log, one hash-chained record per line. The search index is a SQLite FTS5 table that can be deleted and rebuilt at any time. memory_verify compares them and reports drift both ways, because "the index is ahead of the store" is how a memory system loses things while reporting health.

Related MCP server: mcp-evidence-ledger

Tools

Unknown fields are stored verbatim at the top level of the record. Nothing is ever rewritten, so a field this server does not model cannot be lost by it.

Measured against the same probes as everybody else

memory-conformance runs six falsifiable retention probes against a server over MCP stdio. Results:

$ python harness.py --adapter=retained --cmd="python /path/to/retained.py"
ack_readback     PASS   restart        PASS   foreign_field  PASS
foreign_record   PASS   concurrent     PASS   large_record   PASS
0 probe(s) failed

Hooks: the difference between a memory and a library

An MCP server cannot initiate a request - that is in the protocol, not in this implementation. A store reachable only through a tool call is memory the model has to remember to ask for, which means it usually will not. The fix is a host hook that injects a brief at session start:

$ RETAINED_DIR=~/.retained python retained.py --brief --budget=120
memory: 3 record(s)
- [checkpoint] shipped retained 0.2.0 with optional embeddings; retrieval 11/12 h1, 12/12 h3
- [note] conformance: reference 5/6 retention, 0/12 retrieval
- [note] silentfail issue 1: dead-guard is quadratic, 58s of a 134s scan

Newest first, checkpoints before notes, and every brief says how much it left out - a brief that silently truncates would be this project's own defect one level up. --json gives the same content with included, dropped and tokens for a host that wants to budget.

Claude Code (.claude/settings.json):

{ "hooks": { "SessionStart": [ { "hooks": [ { "type": "command",
  "command": "RETAINED_DIR=$HOME/.retained python3 -m retained --brief --budget=800" } ] } ] } }
}

hooks/session-start.sh is a portable version for any harness that can run a command per session. Without a hook, retained is still a correct store - it is just a store nobody consults.

Retrieval: lexical, and optionally semantic

By default the index is SQLite FTS5 and nothing else. Set one environment variable and it becomes a hybrid:

RETAINED_EMBED_URL=http://127.0.0.1:11434/api/embeddings   # any Ollama-style endpoint
RETAINED_EMBED_MODEL=all-minilm                           # 384 dimensions, ~46 MB

Vectors are stored in the index (never in the log - the log is the truth and stays dependency-free), and search fuses the lexical and vector rankings with reciprocal rank fusion. If the embedder is down, writes still land and search still answers lexically: an optional feature must not become a single point of failure.

Measured on the twelve-question retrieval eval in memory-conformance - six questions sharing vocabulary with the memory they want, six paraphrases:

The one lexical miss - "how many machines do we run" against "three hosts: throne, watchtower and pi" - is found at rank 3 with vectors and still not at rank 1, which is a fair picture of a 384- dimension model on a twelve-item corpus: it closes the gap, it does not erase it.

The bug the selftest caught on its first run

The read-back check failed immediately: the file was 183 bytes where 182 were written. Without O_BINARY, Windows opens the file in text mode and turns every LF into CRLF, so the bytes on disk are not the bytes written. One flag, found in the first minute - and it is the same carriage-return family that has broken appends in the estate this came from twice already.

Install and run

pip install git+https://github.com/arm00pv/retained
RETAINED_DIR=~/.retained retained        # MCP server on stdio

Register it in any MCP client as a stdio server:

{ "mcpServers": { "retained": { "command": "retained", "env": { "RETAINED_DIR": "~/.retained" } } } }
}

Limits, stated

• Retrieval is lexical. SQLite FTS5 with bm25 ranking. No embeddings, no semantic search; a memory that needs "find the thing I meant" needs more than this.

• One writer at a time. The append takes an OS lock (flock / msvcrt.locking); concurrent processes serialise rather than merge.

• No hooks. An MCP server cannot initiate a request, so injection has to be driven by the client. Without a hook in your harness, this is a tool the model must remember to call.

• No encryption, no auth, no multi-user scopes beyond the scope field.

• Fourteen self-test checks and six conformance probes are not a proof of correctness. They are the failures that were thought of, and the list above is the rest.

MIT licensed.