LMemory

Workspace-scoped long-term memory MCP. Structured facts (people, things, relations) live in PostgreSQL; searchable narratives, preferences, and image captions live in pgvector. Each deployment serves one active workspace, so a model cannot switch tenants by inventing a workspace_id.

Built for home assistants, personal knowledge bases, and local agents that need to remember before they answer. The default install runs a Chinese embedding model on CPU and does not require an external API key.

Contents

• Why LMemory

• Features

• Architecture

• Quick start

• Connect a client

• MCP tools

• Configuration

• Import from Confluence / GitLab

• Development

• Security

• Documentation

• Contributing

• License

Related MCP server: HAM

Why LMemory

Switch agents. Keep your memory. Agent ecosystems each have their own tools and memory systems, making long-term knowledge difficult to carry from one to another. LMemory separates your data from the agent through a common MCP interface: connect a compatible agent to the same service and keep building on the same memory. The vision is a personal memory layer you can use throughout your life.

flowchart TB
  subgraph Ecosystems["Different agents, separate ecosystems — memory is hard to carry across"]
    A["Today's agent"]
    B["Another ecosystem's agent"]
    C["Your next agent"]
  end

  Skill["LMemory Skill for agents<br/>When to remember · link data · retrieve · correct"]
  MCP["LMemory · Common MCP interface<br/>Store and recall across agent ecosystems"]
  A <-->|MCP| MCP
  B <-->|MCP| MCP
  C <-->|MCP| MCP
  Skill -.->|Guides agents using the memory tools| Ecosystems

  subgraph Memory["Your persistent memory · independent of the agent"]
    Exact["Structured retrieval engine<br/>Exact entity and relation filters"]
    Semantic["Semantic retrieval engine<br/>Keyword + vector hybrid search"]
    Rel[("Relational data · PostgreSQL<br/>People, things, attributes, relations")]
    Vec[("Vector data · pgvector<br/>Facts, preferences, notes, image descriptions")]
    Exact <--> Rel
    Semantic <--> Vec
    Rel <-->|Linked by subject_entity_id| Vec
  end

  MCP <-->|Write / query| Exact
  MCP <-->|Write / search| Semantic
  Memory --> Life["Agents change. Your memory keeps growing.<br/>Designed for lifelong use."]

  classDef agent fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
  classDef core fill:#0f766e,stroke:#115e59,color:#ffffff
  classDef data fill:#f0fdfa,stroke:#14b8a6,color:#134e4a
  classDef guide fill:#fffbeb,stroke:#d97706,color:#78350f
  class A,B,C agent
  class MCP,Life core
  class Exact,Semantic,Rel,Vec data
  class Skill guide

The bundled LMemory Skill teaches agents how to use both retrieval engines and link structured facts with semantic memories. Both data layers live in the same PostgreSQL deployment, with pgvector providing the vector index.

Chat models have no reliable long-term memory. Dumping everything into a vector store makes exact filters hard; tables without embeddings cannot find notes like “where is the passport”.

LMemory writes two linked layers:

Default: dual-write. entity_upsert first, then memory_remember / image_remember with subject_entity_id. Store only a memory when there is no identifiable object; store only an entity when there is nothing useful to embed. Policy: skills/lmemory/SKILL.md.

Features

• Structured objects: people, items, places, organizations, tasks, events; scenario fields in JSONB; no runtime CREATE TABLE / ALTER TABLE

• Typed relations: located_at, owned_by, member_of, assigned_to, related_to

• Long-term memory: facts, preferences, events, commitments, notes; keyword + pgvector search

• Versioned corrections: memory_correct writes a new version instead of leaving two conflicting facts

• Safe forget: forget_prepare → show matches → user confirms → forget_confirm (token expires in 10 minutes)

• Images: originals on the host, descriptions in the vector index; optional local BLIP captions

• Multi-user: private / shared visibility; configurable default user; identity injected by an upstream gateway

• Three embedding modes: local Sentence Transformers (default), in-process hash n-grams, OpenAI-compatible API

• Knowledge import: incremental sync from Confluence pages and GitLab repositories

Architecture

One deployment = one active workspace. Postgres, embedding, and vision stay on the internal Docker network. The host publishes only the MCP port by default.

flowchart LR
  subgraph Clients
    Agent[MCP Client / Agent]
  end
  subgraph Host["Host :18766"]
    MCP[LMemory MCP]
  end
  subgraph Internal["Internal Docker network"]
    PG[(PostgreSQL 16 + pgvector)]
    Emb[Embedding optional]
    Vision[Vision optional]
  end
  subgraph Data["data/"]
    PGData[postgres/]
    Img[images/]
    Models[models/]
  end
  Agent -->|Streamable HTTP| MCP
  MCP --> PG
  MCP --> Img
  MCP -.-> Emb
  MCP -.-> Vision
  PG --> PGData
  Emb --> Models
  Vision --> Models

workspaces
  └─ principals
       ├─ principal_identities
       ├─ entities ── entity_relations
       ├─ memories ── memory_embeddings [vector(1024)]
       └─ media_assets ── media_embeddings
                          └─ image files (data/images)

Vectors are fixed at 1024 dimensions so the HNSW index stays valid. The default Chinese model BAAI/bge-small-zh-v1.5 outputs 512 dimensions; the service L2-normalizes and zero-pads to 1024 without changing cosine ranking.

Quick start

Requirements: Docker Engine and Docker Compose v2. The first run with the default local embedding model downloads weights and takes longer than later starts.

Linux / macOS:

git clone https://github.com/chenhongjun/lmemory.git
cd lmemory
chmod +x scripts/install.sh
./scripts/install.sh

Windows PowerShell:

git clone https://github.com/chenhongjun/lmemory.git
Set-Location lmemory
.\scripts\install.ps1

The installer:

1. Copies .env.example to .env (if missing) and generates a database password

2. Creates data/postgres, data/images, data/models

3. Starts Compose (default local-model profile)

4. Applies versioned migrations in migrations/

5. Waits until http://127.0.0.1:18766/health is ready

Default URLs:

/mcp is not a normal web page. Open it with an MCP client (initialize → tools/list → tools/call), not a browser.

Connect a client

Cursor / generic MCP

Add a Streamable HTTP server:

{
  "mcpServers": {
    "lmemory": {
      "url": "http://127.0.0.1:18766/mcp"
    }
  }
}

Use skills/lmemory/SKILL.md as the agent memory policy: what to dual-write, what not to store, how to correct and forget.

xiaozhi-server

Write main/xiaozhi-server/data/.mcp_server_settings.json:

{
  "mcpServers": {
    "lmemory": {
      "url": "http://host.docker.internal:18766/mcp",
      "transport": "streamable-http"
    }
  }
}

On a shared Linux Docker network, replace host.docker.internal with a reachable address or join the same external network. Enable a tool-capable intent mode, for example:

selected_module:
  Intent: function_call

xiaozhi-server does not load SKILL.md today. Copy the essential rules into the system prompt or add a skill-aware orchestrator.

MCP tools

23 tools. Full parameters: docs/MCP_API.zh-CN.md.

Tools do not accept workspace_id. The active workspace comes from server .env.

Configuration

All keys: .env.example. After editing .env, rerun the installer or docker compose up -d.

# Active workspace. Changing the slug switches logical workspace in the same database.
LMEMORY_WORKSPACE_SLUG=default
LMEMORY_WORKSPACE_NAME=My Workspace
LMEMORY_WORKSPACE_TYPE=personal

# Only MCP is published to the host; default bind is 127.0.0.1
LMEMORY_BIND_ADDRESS=127.0.0.1
LMEMORY_PORT=18766

# Embeddings: local-model | hash | openai-compatible
LMEMORY_EMBEDDING_PROVIDER=local-model
LMEMORY_EMBEDDING_MODEL=BAAI/bge-small-zh-v1.5

# Vision: description-only | local-model | openai-compatible
LMEMORY_VISION_PROVIDER=description-only

All state is on the host (default ./data). Docker named volumes are not used:

data/
  postgres/   PostgreSQL + pgvector
  images/     uploaded originals
  models/     downloaded embedding / vision models
  import/     import sync state (local only, not in Git)

To move hosts: docker compose down, then copy data/ and .env. Do not copy a running data directory. Keep PostgreSQL 16 compatible.

After changing embedding provider or model, call memory_reindex_embeddings with next_offset until has_more is false. Until reindex finishes, search uses only vectors from the current model.

Do not change only LMEMORY_EMBEDDING_DIMENSIONS. The dimension is fixed in the schema; a new migration is required.

Import from Confluence / GitLab

Host-side scripts write through MCP; they do not run in Docker:

pip install -e '.[import]'
python scripts/lmemory-full-import.py
python scripts/lmemory-incremental.py

Set CONFLUENCE_* and GITLAB_* in .env. LMEMORY_URL defaults to http://127.0.0.1:18766/mcp.

• --source confluence / --source gitlab limits the crawl

• --dry-run prints stats without writing

• Fingerprints live in data/import/lmemory-sync-state.json; unchanged docs are skipped on resume

• GitLab uses the API for the default-branch tree; it is not git clone

Development

Python 3.11+:

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\Activate.ps1
pip install -e '.[dev]'
pytest

Validate Compose:

cp .env.example .env
# Replace POSTGRES_PASSWORD=change-me before starting containers
docker compose config

Files in migrations/ run in filename order at startup and are recorded in schema_migrations. Never edit a migration already applied to real data; add a new file.

Repo rules:

• Do not commit .env, data/, or import state files

• Keep secrets in local .env; docs and examples use placeholders

Security

LMemory has no built-in client authentication. MCP binds to 127.0.0.1 by default.

• Trusted local use: keep the default bind

• Remote access: put a TLS + authenticated reverse proxy in front; do not expose the port to the public internet

• user_external_id / identity_provider must come from a trusted gateway (account, device, voiceprint). Never let the model invent them

• Strong isolation (home / company / customer): separate deployments and databases, not workspace switching in one process

• Tools reject secret-like patterns; do not store passwords, OTP codes, tokens, or private keys

• Forget APIs only affect the live database. Backup retention is an operator policy — do not promise instant removal from backups

legal_hold records cannot be forgotten through these tools. Hard delete cascades to vectors; soft delete marks status and clears content so leftover embeddings are ignored.

Documentation

Contributing

Issues and pull requests are welcome. Before you submit:

1. pip install -e '.[dev]' && pytest

2. Keep docs/MCP_API.zh-CN.md in sync with tool changes

3. Schema changes: add a new file under migrations/

4. Do not commit .env, data/, or real secrets

License

MIT License. Copyright (c) 2026 chenhongjun.