Skip to main content
Glama

Recollect

Version history: CHANGELOG.md | What's next: ROADMAP.md | Design notes: docs/DEVLOG.md

Long-term memory for conversations that knows whose memory it is.

Most "AI memory" layers embed text chunks, retrieve the top-k by cosine similarity, and paste them into the prompt. That works until the conversation lasts more than a few sessions, because real dialogue breaks vector search in three specific ways:

  • Misattribution. "My friend's dog is called Pudding" comes back as "your dog Pudding". "My nephew started kindergarten" is filed as a fact about you. A similarity index has no notion of whose memory an event is.

  • Unbounded growth. Every turn is stored forever and ranks equally, so last year's small talk crowds out what actually matters today.

  • Overwrite instead of update. A new statement about the same thing either duplicates or silently replaces the old one, so "what did I say before?" can no longer be answered.

Recollect is a memory layer that treats these as first-class problems, on top of a single SQLite file. It ships as an MCP server, an HTTP API + web UI, and an in-process SDK.

Status: a working prototype (schema v21), not a hosted service. The Python package is still named neuromem internally — the repo rename is in progress. See Roadmap.

Clean public build & API keys

This repository ships with no account, no API key, and no private endpoints. It is a clean build you can clone and publish safely:

  • No keys in source. There is no real sk- key anywhere in the code. Your own key lives only in local data/*.db (settings table, Windows DPAPI-encrypted with a dpapi: prefix), and both data/ and *.db are gitignored — so clone/commit/push never carry it.

  • No personal model params. Model names and endpoints are all overridable via environment variables; see .env.example. With no overrides, a generic local-Ollama default applies (in config.py) that belongs to nobody.

  • First-run guidance. On first open (while the memory store is empty), the overview page shows a banner with the next step for your mode: local-Ollama users see the model-pull commands, cloud users are prompted to enter their own key. It disappears once you have memories, and can be dismissed manually.

Pick one of two setups:

  1. Local Ollama (no key, recommended to start): install Ollama, pull the models below, and it just works.

  2. Cloud OpenAI-compatible endpoint (SenseNova / DeepSeek / DashScope / Moonshot / OpenAI, etc.): in the web "Models" page pick a preset or enter base_url + model names, then paste your own API key. The key stays on your machine only.

To preset model names via env (never the key): copy .env.example to .env and edit the NEUROMEM_* values. Real environment variables take precedence over .env (handy for CI/tests). .env itself is gitignored.


Related MCP server: Munin Memory

Models & where to get them

Two kinds of models are needed; each can be local or cloud (recommended combos at the end):

Role

Purpose

Default (local)

Requirement

LLM extract

turns dialogue into structured memories / affect / profiles

qwen3.5:4b

fast, cheap, emits JSON

LLM answer

generates replies and personas

qwen3:8b

stronger than extract, steadier

Embedding

vectorizes memories; decides retrieval quality

qwen3-embedding:4b

dimension must match the store (2560)

The embedding model is the least safe thing to swap in this system: changing the model or the dimension makes old memories unretrievable, and cosine search silently truncates to the shorter dimension without an error. Unless you are willing to clear and re-ingest the store, stick with local 4B.

Choosing an embedding model

Every recommended model below outputs 2560 dimensions, matching this project's default store. The dimensions parameter is sent automatically; endpoints that don't support it fall back gracefully. Same dimension ≠ same vector space — after switching models you should still rebuild vectors / clear and re-ingest for reliable retrieval.

Option

Model

Dim

Best for

Local Ollama (recommended)

qwen3-embedding:4b

2560 (native)

best Chinese memory retrieval, free, data stays local; this project's default

Local Ollama

qwen3-embedding:8b

2560 (truncated)

#1 on MTEB, highest quality; pick when you have spare resources

SiliconFlow

Qwen/Qwen3-Embedding-4B

2560

one key for cloud LLM + embeddings, good Chinese, CN-direct

SiliconFlow

Qwen/Qwen3-Embedding-8B

2560

best hosted quality at 2560

OpenAI

text-embedding-3-large

2560 (truncated)

all-OpenAI stack; note 3-small caps at 1536 and can't reach 2560

In the web "Models" page, under "Embedding model", click a recommendation to fill the form (fill only, no auto-save), confirm, then click "Save & apply" and verify the returned dimension with "Test embedding".

After switching models, go to the "Data" page and click "Rebuild vectors" — it recomputes embeddings for all memories with the current model and writes them back, without losing memories / personas / conversations. The UI shows live progress (processed count, percentage, elapsed) and can be cancelled; on any error it stops and rolls back entirely, never leaving a half-rebuilt state. Even when old and new models are both 2560-dim, their vector spaces differ, so rebuild after every model switch to keep retrieval accurate.

DashScope text-embedding-v3 and the bge-large family are natively 1024-dim and can't reach 2560, so they are excluded. SenseNova's Token Plan OpenAI-compatible gateway (https://token.sensenova.cn/v1) has no embeddings endpoint; its "Piccolo Embedding" belongs to a separate SenseCore API needing its own key/base_url. So when using SenseNova chat/extract models, keep embeddings on local Ollama — do not call /embeddings on the same base_url.

  1. Install Ollama: https://ollama.com (Windows/macOS/Linux, next-next-finish).

  2. Pull the models (first run downloads them; the embed model is ~2 GB):

    ollama pull qwen3-embedding:4b    # embedding, 2560-dim
    ollama pull qwen3.5:4b            # extraction
    ollama pull qwen3:8b              # answers (optional; 4B is fine for speed)
  3. No key needed — just python -m neuromem serve 8000.

Switching the embedding model requires a matching dimension or old memories become unretrievable. The default store is 2560-dim (4B).

Option B: cloud OpenAI-compatible endpoints (bring your own key)

In the web "Models" page click a preset (it fills base_url + model names), then paste your own API key. All providers below speak the OpenAI protocol — only base_url + key + model name differ:

Provider

Get a key (console)

base_url

Recommended (extract / answer)

Notes

DeepSeek (recommended)

https://platform.deepseek.com

https://api.deepseek.com

deepseek-chat / deepseek-chat

fast, cheap, free credits on signup

SenseNova (SenseTime) (author's own)

https://platform.sensenova.cn

https://token.sensenova.cn/v1

sensenova-6.8-flash-lite / sensenova-6.8-flash

CN-direct; signup credits, dedicated Flash-Lite credit pool

DashScope / Qwen

https://bailian.console.aliyun.com

https://dashscope.aliyuncs.com/compatible-mode/v1

qwen-plus / qwen-max

stable, new-user credits

Moonshot

https://platform.moonshot.cn

https://api.moonshot.cn/v1

moonshot-v1-8k

Kimi series

SiliconFlow

https://cloud.siliconflow.cn

https://api.siliconflow.cn/v1

Qwen/Qwen2.5-7B-Instruct / Qwen/Qwen2.5-72B-Instruct

try many models with one key, free tier

OpenAI

https://platform.openai.com/api-keys

https://api.openai.com/v1

gpt-4o-mini / gpt-4o-mini

needs overseas network + payment

The author's own setup (SenseNova) — the combo this project was tuned with; copy it as-is:

Role

Model ID

Why

Extract

sensenova-6.8-flash-lite

fast, low credit cost; turns dialogue into structured memories

Answer

sensenova-6.8-flash

steadier replies and persona generation

In the web "Models" page click the sensenova preset — it fills base_url and both model names; you only paste your own key. Create one at https://platform.sensenova.cn → console → "API Keys" (sk- prefix).

Getting a key, generally: sign up → open the console's "API Keys" page → create and copy it (usually shown once — save it immediately). Provider names and model IDs change over time; trust what the console shows. If you get model not found, copy the exact ID from the provider's model list.

Recommended combos: start with the full local-Ollama stack (zero cost). For stronger generation, use sensenova-6.8-flash-lite for extract + sensenova-6.8-flash for answer (the author's own setup, one key, CN-direct), or DeepSeek for both. Keep the embedding model local 4B to avoid dimension mismatches and cloud embedding cost.


How it differs

Typical vector memory

Recollect

Attribution

none — text chunks

every event carries subject: self / counterpart / shared / third_party

Forgetting

TTL or nothing

continuous decay act *= exp(-Δt/τ), τ derived from salience, emotion, surprise, confidence, relationships, review count

Updating

overwrite

reconsolidation: old gist + vector kept in memory_versions, re-encoded with the current context, reversible

Counting questions ("how many times…")

top-k can't count

aggregation layer (count / list / relation_count) over the full matching set

Answer safety

none

every candidate's source text is checked against the reply; unsupported sentences are trimmed

Observability

none

adoption log, per-turn trace, mind state, and an A/B lab

Mechanisms

Layer

What it does

Code

L0 constitution

who the agent is, who it serves, tone, hard limits. Human-versioned, read-only at runtime, exempt from reset and decay

identity.py, persona_card

L1 events

extracted events with subject, time, place, affect, surprise

store.py

Concepts / graph

entity clustering, concept induction, diffusion activation at recall

consolidator.py, graph.py

Recall

vector + entity + graph diffusion + fusion + credibility tiers, with a relevance floor

recall.py

Forgetting

activation decay + salience compensation + archiving

forgetting.py

Reconsolidation

versioned rewrite on feedback, 1h cooldown, 12-version cap

adoption.py

Affect

8-dimensional discrete emotion + slow mood + per-person affect

affect.py

Aggregation

answers count/list/relation questions from the full set instead of top-k

aggregate.py

Fact guard

trims sentences the evidence doesn't support; refuses instead of inventing

agent/companion.py

conversation turn
      │
      ▼
 Brain.turn()  ──►  recall  (vector + entity + graph + fusion)
      │                    │
      │                    ▼
      │              ContextPack  ──►  LLM  ──►  structured output
      │                                              │
      ▼                                              ▼
 Brain.commit()  ◄──  adopted ids / new facts / affect / mind state
      │
      ├─► events (+subject, surprise, affect)
      ├─► adoption_log  ──► reconsolidation (memory_versions)
      └─► trace (latency, candidates, used_ids, think tokens)

Person-to-person relations: extraction limits & manual entry

A "relation" (A —relation→ B, with strength and evidence) lives in its own relations table. It is distinct from a personas card and from edges.involves (an event mentioning a person). Remembering someone, having their portrait, and even 200 events involving them does not imply a person-to-person link exists — that line is separate data.

How relations are made (automatically, right after an event is written, in store.py):

  • ✅ Auto-linked: sentences with a judgment structure — "I have a friend named Lin Wan", "Zhou Ye was my mentor when I joined", "Zhiwei runs the B&B". Per-sentence LLM extraction plus a local pattern fallback.

  • ❌ Missed: plain statements where the relation word appears but there is no "X is / is with Y's " structure — e.g. "my cousin says he works even later than I do", "my older sister studied abroad in the UK; she said it rains a lot there". Neither the model nor the rules classify these as relations.

  • Measured: after clearing all 8 stored relations and replaying their source sentences, auto-extraction recovered 6/8 — the two misses were exactly the plain statements above. Relation extraction is therefore semi-automatic, not fully automatic.

This is a capability boundary, not a bug. Name resolution works: saying "I'm Amu" resolves correctly to the counterpart (both resolve_persona('Amu') and "you" point to the same entity), so it never links to the wrong person — it just can't parse the relation inside those statement-style sentences yet.

When to add manually: if you mentioned someone but no line appeared under the Relations tab, add it there — pick A and B, enter the relation word (e.g. cousin), click "Add relation". Existing relations can be edited or deleted there too. This is the standard fix for missed statements; there's no need to repeat the sentence.

Other notes: relations need textual evidence — model-invented ones are rejected by a hard check. Generic terms ("friend/coworker/roommate") don't create named people unless a person of that name already exists.

Quick start

Requires Python 3.11+. The default setup uses Ollama locally (no API key needed); you can point the chat/embedding models at any OpenAI-compatible endpoint in the web UI using your own key. Model names are configurable via .env — see Clean public build.

pip install -r requirements.txt
ollama pull qwen3-embedding:4b        # default local embed model (2560-dim, strong on Chinese); configurable

python -m neuromem init               # idempotent: tables + defaults
python tools/seed_demo.py             # optional: 12 fictional memories to look at
python -m neuromem serve 8000         # http://127.0.0.1:8000

On first login, create an admin account in the browser. If you later switch to a cloud model without a key, the overview page will guide you to add your own.

Chat with memory in the web UI, or call the API:

curl -X POST http://127.0.0.1:8000/api/live -H "Content-Type: application/json" \
     -d '{"query":"我朋友养的狗叫什么?"}'

Read-only recall (does not write memory) is POST /api/ask; MCP's memory_retrieve uses the same path.

MCP

{
  "mcpServers": {
    "recollect": {
      "command": "python",
      "args": ["/absolute/path/to/tools/mcp_server.py"]
    }
  }
}

Ten tools: memory_retrieve, memory_remember, memory_ask, memory_person, memory_recent, memory_status, memory_state, memory_turn, memory_surface, memory_feedback.

memory_surface returns memories worth bringing up unprompted; memory_feedback reports which candidates were actually used, which is what triggers reconsolidation (exposure ≠ adoption).

In-process (for companion agents)

from neuromem.brain import Brain

brain = Brain()
pack = brain.turn("我叫小林,在杭州做设计")
# ... generate a reply from pack.text ...
brain.commit(pack, used_ids=[...], remember=[...])

agent/companion.py is a full reference implementation: it decides used_ids itself, writes facts, and records a trace per turn.

Does it actually work?

Honesty first: there is no public benchmark number yet (LoCoMo / LongMemEval are the next milestone, see Roadmap). What exists today is engineering verification:

  • 364 unit tests, all green on a fresh clone with no local database (pytest tests/unit)

  • an 8-question regression harness with expected-evidence checks (tools/eval_regression.py)

  • ~150 turns of real usage traces from the web channel, used to find and fix four recall bugs and one attribution bug (see docs/DEVLOG.md)

  • ablation via NEUROMEM_LLM_THINK=on, AGGREGATE_LAYER=0, etc. — mechanisms are switchable, so they can be measured rather than asserted

Repository layout

neuromem/     memory core (store, recall, forgetting, reconsolidation, affect, aggregate)
agent/        reference companion agent + prompts + trace
tools/        MCP server, evaluation harnesses, probes, migrations
web/          single-page management UI
tests/        364 unit tests
docs/         architecture notes and the development log

Roadmap

  • LoCoMo / LongMemEval results, with an ablation table

  • Docker one-liner + seeded demo database (no Ollama required to click around)

  • Embedding fallback (hashing / small model) so the first run isn't a 2.5 GB download

  • Package rename (neuromem → the new name) and PyPI release

  • English + Chinese docs split, architecture diagrams

Contributing

Issues and PRs are welcome, especially: Chinese long-conversation test cases, attribution edge cases (kinship, pets, groups), and benchmark adapters. Please run pytest tests/unit before sending a PR — the suite is the contract.

Support

If Recollect has been useful, the web UI has a "Support" page with QR codes (entirely optional — all features are free). QR images live in web/donate/ (wechat.png / alipay.png / coffee.png) and are committed with the repo, so anyone who clones can see them. Prefer to contribute instead? Issues, Chinese long-conversation test cases, and benchmark adapters are worth far more — see Contributing above.

License

Apache-2.0. See LICENSE.


中文说明 · 开发日志 docs/DEVLOG.md

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • -
      license
      Not graded
      quality
      Not graded
      maintenance
      Provides persistent local memory functionality for AI assistants, enabling them to store, retrieve, and search contextual information across conversations with SQLite-based full-text search. All data stays private on your machine while dramatically improving context retention and personalized assistance.
      3
      -
    • A
      license
      A
      quality
      D
      maintenance
      Enables AI assistants to maintain long-term memory by logging and retrieving facts and decisions in a SQLite database, with relevance ranking and full history tracking.
      7
      MIT
    • A
      license
      B
      quality
      B
      maintenance
      Enables AI assistants to retain atomic facts, decisions, and episodic lessons in a persistent shared SQLite graph memory, then recall them through hybrid BM25 plus dense-vector search with spreading activation and progressively packed context. Also supports reflection-based belief revision so stale or contradictory memories are superseded rather than returned alongside current ones.
      4
      MIT