<div align="center">

# EvoMem-MCP

### Memory for AI agents that knows when you changed your mind.

**English** · [Español](README.es.md)

</div>

---

You tell your coding agent: *"we're using Postgres for the ledger."*
Two weeks later: *"we moved the ledger to SQLite, Postgres was overkill."*

A month after that you ask *"what database does the ledger use?"* Your agent's memory
holds **both** notes, and the old one matches the question better. So the agent
**recommends Postgres again.**

This is the default failure mode of agent memory: a pile of facts with no idea which
ones are still true. **EvoMem-MCP** is a local [MCP](https://modelcontextprotocol.io)
memory server that stores decisions as a **history**. Every new decision is linked to
the one it replaces, the old one is kept but marked `superseded`, and `recall` always
resolves to the version currently in force.

```
recall("ledger postgres")          # the query matches the OLD note best...
→ change: "We moved the ledger from Postgres to SQLite."     status: active
  └─ supersedes: "We picked Postgres for the ledger."        status: superseded
```

## The result

We built a benchmark to test one hypothesis: **does a memory that links decisions to
the ones they replace stop an agent from acting on obsolete decisions?**

The metric is **ODR (Obsolete Decision Rate)**: the share of questions where the agent
acts on a decision that was already replaced. We used 120 frozen scenarios, with 2 to 6
versions of a decision per topic plus distractors. The judge is a local **27B LLM**
(Bonsai 2 27B, temperature 0). It reads what memory returns and names the option it
would act on.

| memory | ODR ↓ | 95% CI |
|---|---|---|
| flat (all notes, no dates) | 0.42 | 0.33–0.51 |
| flat **with dates** (the fair baseline) | 0.20 | 0.14–0.28 |
| **EvoMem: links set by the agent** (`remember(supersedes=…)`) | **0.00** | 0.00–0.03 |
| **EvoMem: links detected automatically, LLM linker** | **0.00** | 0.00–0.03 |
| EvoMem: links detected automatically, heuristic linker | 0.008 | 0.00–0.05 |

Paired per scenario against *flat with dates*, EvoMem wins **24 scenarios and loses 0**
(exact McNemar, p ≈ 1e-7).

### The surprising part: dates are not enough

The memory was **flat with dates**, and the current version was right there in the
context with its date in **111 of 120** scenarios. **The 27B model still acted on an
obsolete decision in 15% of them.** Once the old note was explicitly marked
`status: superseded`, that error went to **0**.

So "just show timestamps and let the model figure it out" doesn't work reliably. An
explicit "this was replaced" marker does.

## How it works

```mermaid
flowchart LR
    A[Agent] -- "remember(text, supersedes?)" --> S[EvoMem server]
    S --> C["Cronista<br/>extracts trigger · change · consequences"]
    S --> L["Linker (optional)<br/>which active decision does this replace?"]
    C --> DB[("SQLite<br/>nodes + raw text + FTS5/BM25")]
    L --> DB
    A -- "recall(query)" --> S
    DB -- "every hit resolved to the<br/>head of its chain" --> A
```

- **Every note becomes a `MemoryNode`.** Its narrative fields are `trigger` (why),
  `change` (what was decided), `alternatives_tried`, `consequences` and `summary`. The
  raw text is always kept and indexed, so recall never depends on the extraction
  being perfect.
- **`supersede` never deletes.** The old node is kept with `status: superseded` and a
  link to its replacement. `recall(include_history=True)` returns the whole chain,
  newest to oldest, so the agent can see *why* things changed.
- **`recall` resolves to the chain head.** A query that only matches an old version
  still returns the current one. This is what the benchmark measures.
- **Who creates the links?**
  - **The agent:** `remember(text, supersedes=id)`. This is the most reliable option.
  - **The server**, with `--auto-supersede`, when the agent doesn't pass an id:
    - `llm` uses any OpenAI-compatible endpoint, local or remote;
    - `heuristic` needs a change word plus overlapping vocabulary. It has no extra
      cost, but it is not safe on its own (see below).
- **Everything is local:** one SQLite file, no cloud, no embeddings service.

## Quickstart

```bash
git clone https://github.com/agunet/evomem-mcp.git && cd evomem-mcp
pip install -e .                      # or: uvx --from . evomem-mcp --db ~/.evomem/evomem.db
evomem-mcp --db ~/.evomem/evomem.db   # lite mode: no GPU, ~7 ms per remember
```

**Claude Code:**

```bash
claude mcp add evomem -- evomem-mcp --db ~/.evomem/evomem.db
```

**Claude Desktop / any MCP client:**

```json
{ "mcpServers": { "evomem": {
    "command": "evomem-mcp",
    "args": ["--db", "/home/you/.evomem/evomem.db"] } } }
```

**Automatic supersession** with a local model (here llama.cpp; any OpenAI-compatible
API works):

```bash
EVOMEM_LINKER_API_KEY=... evomem-mcp --db ~/.evomem/evomem.db \
    --auto-supersede llm --linker-model <model> --linker-base-url http://localhost:8080/v1
```

### Tools

| tool | what it does |
|---|---|
| `remember(raw_text, tags?, session?, relations?, supersedes?)` | Stores a note as a `MemoryNode`. With `supersedes`, the note replaces that decision (or the current head of its chain). |
| `recall(query, k=5, include_history=False, history_depth=5)` | BM25 search over the fields and raw text. Each hit is resolved to its current version, optionally with its history. |
| `get_node(node_id)` | Fetches one node. |
| `forget(node_id, purge=False)` | Removes a node from recall but keeps its history. `purge` deletes the node and keeps the raw text. |
| `stats()` | Counts nodes by status. |

## Reproduce it

The scenarios are generated from a fixed seed, so you get exactly the same 120. The
deterministic judge runs in **about a minute on CPU**:

```bash
python make_odr_scenarios.py --out data/odr_scenarios.jsonl --n 120
python benchmark_decisiones.py --scenarios data/odr_scenarios.jsonl --auto heuristic
```

The LLM-judge numbers above need a judge model behind an OpenAI-compatible endpoint:

```bash
python benchmark_decisiones.py --scenarios data/odr_scenarios.jsonl --judge llm \
    --judge-model <model> --judge-base-url http://localhost:8080/v1 --auto heuristic,llm
```

There is also a second benchmark, **VDR**, which measures the opposite risk:
*does automatic linking hide decisions that are still valid?*

```bash
python make_odr_scenarios.py --vdr --out data/odr_vdr.jsonl --n 80 --seed 11
```

## The cost of automatic linking: VDR

A linker that is too eager is dangerous. Suppose you note *"we switched the **staging**
ledger to SQLite"*. If the linker treats that as replacing the **production** ledger
decision, the production decision disappears from recall, even though it is still
valid.

The VDR set has 80 scenarios where each component has a decision in another scope
(staging, analytics, EU region…) that stays valid the whole time. Half of them use
change words on purpose to fool the linker.

| linker | link precision | still-valid decisions it hides (out of 80) |
|---|---|---|
| perfect links (reference) | 1.00 | — |
| **LLM** (Bonsai 2 27B) | **0.95** | **1** |
| heuristic | 0.70 (0.56 with change words) | **8** |

**Verdict:** in these scenarios the LLM linker works without a human in the loop, at
the cost of about 1.5 s per `remember` with a local 27B. The heuristic linker should
not run unsupervised.

## What this does *not* prove (yet)

We'd rather you hear the limits from us:

- **The scenarios are synthetic**, with one judge model and one seed. Real agent logs
  are messier. Evaluating on real agent history is on the roadmap.
- **The "story" part is still untested.** With `history_depth=0` (only the current
  version) ODR is already 0, so the benefit measured here comes from *filtering what
  is obsolete*. It does not yet show that seeing the *why* behind each change helps.
  That needs scenarios where the reason a decision was abandoned matters ("should we
  try Kafka?" when Kafka was already dropped for a reason that still holds).
- **Scope confusion is a separate problem.** Even with perfect links, the judge applies
  a production decision to analytics in 21–30% of VDR queries. An explicit `scope`
  field is planned.
- **The heuristic linker misses paraphrases.** It relies on overlapping words. Pass
  `supersedes` yourself, or use the LLM linker.

## Side quest: a tiny model as the note-taker

The "cronista" (note-taker) that turns text into fields comes in three versions:

| cronista | how | latency (p50, RTX 3060) |
|---|---|---|
| `lite` (default) | sentence heuristics, no GPU | ~7 ms |
| `ncm` | [NCM](https://github.com/HazyResearch/ncm) 88M base model writes the summary | 207 ms |
| `ncm` + structural LoRA | 88M model + a 2.4M-parameter LoRA we trained on 120k synthetic examples | 359 ms |

What we learned, including the negative results:

- **Fine-tuning for generic summarization doesn't help.** On XSum the LoRA does not
  beat the base model (ROUGE-1 34.3 vs 34.9). For general summaries the model is
  already at its ceiling, so fine-tuning is only worth it for structured extraction.
- **Structured extraction works.** The trained LoRA reaches ROUGE 88–93 on
  trigger / change / consequences. On a validation set with different wording that it
  never saw it scores almost the same, so it generalizes.
- **The generated summary invents things.** It gets the service and the numbers right
  only 26–37% of the time. We fixed two dataset bugs and retrained, and it still
  invents. The likely cause is that the model's discrete decoding corrupts literal
  names and numbers.
- **So we stopped generating the summary.** By default it is assembled from the
  extracted fields, after matching each one to the closest sentence in the source.
  That summary is correct **88–93%** of the time and is also the fastest option.
- The goal of under 100 ms per `remember` on the GPU path is **not met yet** (359 ms).

The full experiment log, with every table, ablation, dead end and dataset bug, is in
[`docs/LAB_NOTEBOOK.md`](docs/LAB_NOTEBOOK.md) (in Spanish).

## Roadmap

- [ ] Linker that can **abstain**: when unsure, return the candidates to the agent
      instead of guessing.
- [ ] A cheaper linker: a small model, called only when the note contains a change word.
- [ ] An explicit `scope` field, to fix production vs staging vs analytics mix-ups.
- [ ] Hybrid search (BM25 + embeddings) for notes that say the same thing in other
      words.
- [ ] **Trajectory scenarios**, to test whether seeing *why* a decision was dropped
      helps, beyond filtering what is obsolete.
- [ ] Evaluation on real agent histories.
- [ ] PyPI release.

## Status

`v0.2.1`, a research prototype. The MCP server works end to end and the tests pass
(`python test_smoke.py`, `python smoke_mcp.py`). The API may still change.

## License

[MIT](LICENSE)

## Credits

The NCM (Next-Concept Modeling) base model is by [HazyResearch](https://github.com/HazyResearch/ncm). EvoMem's history design
is inspired by hierarchical, versioned memory (CoMeT-style `supersede` / `merge`).