Recollect
Supports local Ollama models for structured memory extraction, answer/persona generation, and embeddings, with configurable model names and no API key required.
Supports OpenAI and OpenAI-compatible cloud endpoints, allowing users to configure base URL, model names, and their own API key for memory extraction, answer generation, and embeddings.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Recollect
Version history: CHANGELOG.md | What's next: ROADMAP.md | Design notes: docs/DEVLOG.md
Long-term memory for conversations that knows whose memory it is.
Most "AI memory" layers embed text chunks, retrieve the top-k by cosine similarity, and paste them into the prompt. That works until the conversation lasts more than a few sessions, because real dialogue breaks vector search in three specific ways:
Misattribution. "My friend's dog is called Pudding" comes back as "your dog Pudding". "My nephew started kindergarten" is filed as a fact about you. A similarity index has no notion of whose memory an event is.
Unbounded growth. Every turn is stored forever and ranks equally, so last year's small talk crowds out what actually matters today.
Overwrite instead of update. A new statement about the same thing either duplicates or silently replaces the old one, so "what did I say before?" can no longer be answered.
Recollect is a memory layer that treats these as first-class problems, on top of a single SQLite file. It ships as an MCP server, an HTTP API + web UI, and an in-process SDK.
Status: a working prototype (schema v21), not a hosted service. The Python package is still named
neuromeminternally — the repo rename is in progress. See Roadmap.
Clean public build & API keys
This repository ships with no account, no API key, and no private endpoints. It is a clean build you can clone and publish safely:
No keys in source. There is no real
sk-key anywhere in the code. Your own key lives only in localdata/*.db(settingstable, Windows DPAPI-encrypted with adpapi:prefix), and bothdata/and*.dbare gitignored — so clone/commit/push never carry it.No personal model params. Model names and endpoints are all overridable via environment variables; see
.env.example. With no overrides, a generic local-Ollama default applies (inconfig.py) that belongs to nobody.First-run guidance. On first open (while the memory store is empty), the overview page shows a banner with the next step for your mode: local-Ollama users see the model-pull commands, cloud users are prompted to enter their own key. It disappears once you have memories, and can be dismissed manually.
Pick one of two setups:
Local Ollama (no key, recommended to start): install Ollama, pull the models below, and it just works.
Cloud OpenAI-compatible endpoint (SenseNova / DeepSeek / DashScope / Moonshot / OpenAI, etc.): in the web "Models" page pick a preset or enter base_url + model names, then paste your own API key. The key stays on your machine only.
To preset model names via env (never the key): copy
.env.exampleto.envand edit theNEUROMEM_*values. Real environment variables take precedence over.env(handy for CI/tests)..envitself is gitignored.
Related MCP server: Munin Memory
Models & where to get them
Two kinds of models are needed; each can be local or cloud (recommended combos at the end):
Role | Purpose | Default (local) | Requirement |
LLM extract | turns dialogue into structured memories / affect / profiles |
| fast, cheap, emits JSON |
LLM answer | generates replies and personas |
| stronger than extract, steadier |
Embedding | vectorizes memories; decides retrieval quality |
| dimension must match the store (2560) |
The embedding model is the least safe thing to swap in this system: changing the model or the dimension makes old memories unretrievable, and cosine search silently truncates to the shorter dimension without an error. Unless you are willing to clear and re-ingest the store, stick with local 4B.
Choosing an embedding model
Every recommended model below outputs 2560 dimensions, matching this project's default store. The
dimensionsparameter is sent automatically; endpoints that don't support it fall back gracefully. Same dimension ≠ same vector space — after switching models you should still rebuild vectors / clear and re-ingest for reliable retrieval.
Option | Model | Dim | Best for |
Local Ollama (recommended) |
| 2560 (native) | best Chinese memory retrieval, free, data stays local; this project's default |
Local Ollama |
| 2560 (truncated) | #1 on MTEB, highest quality; pick when you have spare resources |
SiliconFlow |
| 2560 | one key for cloud LLM + embeddings, good Chinese, CN-direct |
SiliconFlow |
| 2560 | best hosted quality at 2560 |
OpenAI |
| 2560 (truncated) | all-OpenAI stack; note |
In the web "Models" page, under "Embedding model", click a recommendation to fill the form (fill only, no auto-save), confirm, then click "Save & apply" and verify the returned dimension with "Test embedding".
After switching models, go to the "Data" page and click "Rebuild vectors" — it recomputes embeddings for all memories with the current model and writes them back, without losing memories / personas / conversations. The UI shows live progress (processed count, percentage, elapsed) and can be cancelled; on any error it stops and rolls back entirely, never leaving a half-rebuilt state. Even when old and new models are both 2560-dim, their vector spaces differ, so rebuild after every model switch to keep retrieval accurate.
DashScope
text-embedding-v3and thebge-largefamily are natively 1024-dim and can't reach 2560, so they are excluded. SenseNova's Token Plan OpenAI-compatible gateway (https://token.sensenova.cn/v1) has no embeddings endpoint; its "Piccolo Embedding" belongs to a separate SenseCore API needing its own key/base_url. So when using SenseNova chat/extract models, keep embeddings on local Ollama — do not call/embeddingson the same base_url.
Option A: local Ollama (free, no key — recommended to start)
Install Ollama: https://ollama.com (Windows/macOS/Linux, next-next-finish).
Pull the models (first run downloads them; the embed model is ~2 GB):
ollama pull qwen3-embedding:4b # embedding, 2560-dim ollama pull qwen3.5:4b # extraction ollama pull qwen3:8b # answers (optional; 4B is fine for speed)No key needed — just
python -m neuromem serve 8000.
Switching the embedding model requires a matching dimension or old memories become unretrievable. The default store is 2560-dim (4B).
Option B: cloud OpenAI-compatible endpoints (bring your own key)
In the web "Models" page click a preset (it fills base_url + model names), then paste your own API key. All providers below speak the OpenAI protocol — only base_url + key + model name differ:
Provider | Get a key (console) | base_url | Recommended (extract / answer) | Notes |
DeepSeek (recommended) |
|
| fast, cheap, free credits on signup | |
SenseNova (SenseTime) (author's own) |
|
| CN-direct; signup credits, dedicated Flash-Lite credit pool | |
DashScope / Qwen |
|
| stable, new-user credits | |
Moonshot |
|
| Kimi series | |
SiliconFlow |
|
| try many models with one key, free tier | |
OpenAI |
|
| needs overseas network + payment |
The author's own setup (SenseNova) — the combo this project was tuned with; copy it as-is:
Role | Model ID | Why |
Extract |
| fast, low credit cost; turns dialogue into structured memories |
Answer |
| steadier replies and persona generation |
In the web "Models" page click the sensenova preset — it fills base_url and both model names; you only paste your own key. Create one at https://platform.sensenova.cn → console → "API Keys" (sk- prefix).
Getting a key, generally: sign up → open the console's "API Keys" page → create and copy it (usually shown once — save it immediately). Provider names and model IDs change over time; trust what the console shows. If you get
model not found, copy the exact ID from the provider's model list.
Recommended combos: start with the full local-Ollama stack (zero cost). For stronger generation, use sensenova-6.8-flash-lite for extract + sensenova-6.8-flash for answer (the author's own setup, one key, CN-direct), or DeepSeek for both. Keep the embedding model local 4B to avoid dimension mismatches and cloud embedding cost.
How it differs
Typical vector memory | Recollect | |
Attribution | none — text chunks | every event carries |
Forgetting | TTL or nothing | continuous decay |
Updating | overwrite | reconsolidation: old gist + vector kept in |
Counting questions ("how many times…") | top-k can't count | aggregation layer ( |
Answer safety | none | every candidate's source text is checked against the reply; unsupported sentences are trimmed |
Observability | none | adoption log, per-turn trace, mind state, and an A/B lab |
Mechanisms
Layer | What it does | Code |
L0 constitution | who the agent is, who it serves, tone, hard limits. Human-versioned, read-only at runtime, exempt from reset and decay |
|
L1 events | extracted events with |
|
Concepts / graph | entity clustering, concept induction, diffusion activation at recall |
|
Recall | vector + entity + graph diffusion + fusion + credibility tiers, with a relevance floor |
|
Forgetting | activation decay + salience compensation + archiving |
|
Reconsolidation | versioned rewrite on feedback, 1h cooldown, 12-version cap |
|
Affect | 8-dimensional discrete emotion + slow mood + per-person affect |
|
Aggregation | answers count/list/relation questions from the full set instead of top-k |
|
Fact guard | trims sentences the evidence doesn't support; refuses instead of inventing |
|
conversation turn
│
▼
Brain.turn() ──► recall (vector + entity + graph + fusion)
│ │
│ ▼
│ ContextPack ──► LLM ──► structured output
│ │
▼ ▼
Brain.commit() ◄── adopted ids / new facts / affect / mind state
│
├─► events (+subject, surprise, affect)
├─► adoption_log ──► reconsolidation (memory_versions)
└─► trace (latency, candidates, used_ids, think tokens)Person-to-person relations: extraction limits & manual entry
A "relation" (A —relation→ B, with strength and evidence) lives in its own relations table. It is distinct from a personas card and from edges.involves (an event mentioning a person). Remembering someone, having their portrait, and even 200 events involving them does not imply a person-to-person link exists — that line is separate data.
How relations are made (automatically, right after an event is written, in store.py):
✅ Auto-linked: sentences with a judgment structure — "I have a friend named Lin Wan", "Zhou Ye was my mentor when I joined", "Zhiwei runs the B&B". Per-sentence LLM extraction plus a local pattern fallback.
❌ Missed: plain statements where the relation word appears but there is no "X is / is with Y's " structure — e.g. "my cousin says he works even later than I do", "my older sister studied abroad in the UK; she said it rains a lot there". Neither the model nor the rules classify these as relations.
Measured: after clearing all 8 stored relations and replaying their source sentences, auto-extraction recovered 6/8 — the two misses were exactly the plain statements above. Relation extraction is therefore semi-automatic, not fully automatic.
This is a capability boundary, not a bug. Name resolution works: saying "I'm Amu" resolves correctly to the counterpart (both resolve_persona('Amu') and "you" point to the same entity), so it never links to the wrong person — it just can't parse the relation inside those statement-style sentences yet.
When to add manually: if you mentioned someone but no line appeared under the Relations tab, add it there — pick A and B, enter the relation word (e.g. cousin), click "Add relation". Existing relations can be edited or deleted there too. This is the standard fix for missed statements; there's no need to repeat the sentence.
Other notes: relations need textual evidence — model-invented ones are rejected by a hard check. Generic terms ("friend/coworker/roommate") don't create named people unless a person of that name already exists.
Quick start
Requires Python 3.11+. The default setup uses Ollama locally (no API key needed); you can point the chat/embedding models at any OpenAI-compatible endpoint in the web UI using your own key. Model names are configurable via .env — see Clean public build.
pip install -r requirements.txt
ollama pull qwen3-embedding:4b # default local embed model (2560-dim, strong on Chinese); configurable
python -m neuromem init # idempotent: tables + defaults
python tools/seed_demo.py # optional: 12 fictional memories to look at
python -m neuromem serve 8000 # http://127.0.0.1:8000On first login, create an admin account in the browser. If you later switch to a cloud model without a key, the overview page will guide you to add your own.
Chat with memory in the web UI, or call the API:
curl -X POST http://127.0.0.1:8000/api/live -H "Content-Type: application/json" \
-d '{"query":"我朋友养的狗叫什么?"}'Read-only recall (does not write memory) is POST /api/ask; MCP's memory_retrieve uses the same path.
MCP
{
"mcpServers": {
"recollect": {
"command": "python",
"args": ["/absolute/path/to/tools/mcp_server.py"]
}
}
}Ten tools: memory_retrieve, memory_remember, memory_ask, memory_person, memory_recent, memory_status, memory_state, memory_turn, memory_surface, memory_feedback.
memory_surface returns memories worth bringing up unprompted; memory_feedback reports which candidates were actually used, which is what triggers reconsolidation (exposure ≠ adoption).
In-process (for companion agents)
from neuromem.brain import Brain
brain = Brain()
pack = brain.turn("我叫小林,在杭州做设计")
# ... generate a reply from pack.text ...
brain.commit(pack, used_ids=[...], remember=[...])agent/companion.py is a full reference implementation: it decides used_ids itself, writes facts, and records a trace per turn.
Does it actually work?
Honesty first: there is no public benchmark number yet (LoCoMo / LongMemEval are the next milestone, see Roadmap). What exists today is engineering verification:
364 unit tests, all green on a fresh clone with no local database (
pytest tests/unit)an 8-question regression harness with expected-evidence checks (
tools/eval_regression.py)~150 turns of real usage traces from the web channel, used to find and fix four recall bugs and one attribution bug (see docs/DEVLOG.md)
ablation via
NEUROMEM_LLM_THINK=on,AGGREGATE_LAYER=0, etc. — mechanisms are switchable, so they can be measured rather than asserted
Repository layout
neuromem/ memory core (store, recall, forgetting, reconsolidation, affect, aggregate)
agent/ reference companion agent + prompts + trace
tools/ MCP server, evaluation harnesses, probes, migrations
web/ single-page management UI
tests/ 364 unit tests
docs/ architecture notes and the development logRoadmap
LoCoMo / LongMemEval results, with an ablation table
Docker one-liner + seeded demo database (no Ollama required to click around)
Embedding fallback (hashing / small model) so the first run isn't a 2.5 GB download
Package rename (
neuromem→ the new name) and PyPI releaseEnglish + Chinese docs split, architecture diagrams
Contributing
Issues and PRs are welcome, especially: Chinese long-conversation test cases, attribution edge cases (kinship, pets, groups), and benchmark adapters. Please run pytest tests/unit before sending a PR — the suite is the contract.
Support
If Recollect has been useful, the web UI has a "Support" page with QR codes (entirely optional — all features are free). QR images live in web/donate/ (wechat.png / alipay.png / coffee.png) and are committed with the repo, so anyone who clones can see them. Prefer to contribute instead? Issues, Chinese long-conversation test cases, and benchmark adapters are worth far more — see Contributing above.
License
Apache-2.0. See LICENSE.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Durable, revision-pinned memory storage and retrieval for AI conversations.
Long-term memory for AI assistants. Isolated per-user storage, recall across conversations.
Persistent AI memory with semantic search, conflict detection, and ticketing.
Persistent memory layer for AI clients: store, recall, and correct context across tools.
Related MCP Servers
- -licenseNot gradedqualityNot gradedmaintenanceProvides persistent local memory functionality for AI assistants, enabling them to store, retrieve, and search contextual information across conversations with SQLite-based full-text search. All data stays private on your machine while dramatically improving context retention and personalized assistance.3-
- AlicenseNot gradedqualityBmaintenanceEnables AI assistants to have persistent, self-hosted memory across sessions and devices, storing context in a SQLite database you control.1MIT
- AlicenseAqualityDmaintenanceEnables AI assistants to maintain long-term memory by logging and retrieving facts and decisions in a SQLite database, with relevance ranking and full history tracking.7MIT
- AlicenseBqualityBmaintenanceEnables AI assistants to retain atomic facts, decisions, and episodic lessons in a persistent shared SQLite graph memory, then recall them through hybrid BM25 plus dense-vector search with spreading activation and progressively packed context. Also supports reflection-based belief revision so stale or contradictory memories are superseded rather than returned alongside current ones.4MIT