Dense Mem

Dense-Mem gives MCP clients a durable memory layer with provenance, typed claims and facts, verification gates, server-side embeddings, recall, team isolation, REST/OpenAPI, a user portal with graph inspection, and a token-protected control portal. The host LLM owns conversation and judgment; Dense-Mem owns durable memory state and returns structured outcomes the host can explain to users.

Under the hood, Dense-Mem is a standalone HTTP MCP memory server. HTTP MCP is the v1 supported MCP transport and is served at /mcp from the main HTTP process.

Dense-Mem is part of the research preprint Governed Enterprise AI Memory Beyond RAG: From Vector Retrieval to Permissioned Knowledge Graphs.

Project Intro

Related MCP server: MemHeaven

Try the Hosted Demo

Create a temporary isolated team at https://demo-dense-mem.markhuang.ai and test Dense-Mem before self-hosting.

Why Dense-Mem?

AI agents need memory that can be trusted later, not only text that can be retrieved later.

• Evidence is first-class. Memories start as source fragments before they become claims or facts.

• Facts pass through typed claims, verification, and promotion gates.

• Comparable conflicts become clarifications[]; Dense-Mem does not silently overwrite active facts.

• The host LLM stays responsible for extracting candidates and asking the user questions. Dense-Mem stays responsible for durable state, gates, audit metadata, and recall.

• Operators keep control of storage, team/profile isolation, API keys, and data egress boundaries.

60-Second Quickstart

Download the base local-only compose example and env template, set the required secrets, and start Dense-Mem:

mkdir dense-mem-local
cd dense-mem-local

curl -fsSLo docker-compose.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/docker-compose.base.yml
curl -fsSLo .env.example \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/.env.example

cp .env.example .env
# Fill in POSTGRES_PASSWORD, NEO4J_PASSWORD, CONTROL_PORTAL_TOKEN, and AI_API_KEY.
${EDITOR:-vi} .env

docker compose up -d
docker compose exec server /app/provision-team --name "primary-memory"

The base compose example provisions Postgres, neo4j:5.26-community with the Neo4j Graph Data Science plugin, and the Dense-Mem server. It exposes only local host ports:

MCP/API:        http://127.0.0.1:8080/mcp
User portal:    http://127.0.0.1:8080/ui
Control portal: http://127.0.0.1:8090/

The user portal includes recall, facts, claims, fragments, communities, dreams, and a bounded graph explorer for seeing facts, claims, evidence fragments, and dream hypotheses as connected memory. The graph view is a read-scoped UI endpoint, not a raw Cypher or unrestricted traversal API.

Cold image pulls can take longer than 60 seconds. Redis and public HTTPS are intentionally omitted from the base example; use the expert example when you need those deployment options.

The server requires a complete embedding configuration at startup: AI_API_URL, AI_API_KEY, AI_API_EMBEDDING_MODEL, and AI_API_EMBEDDING_DIMENSIONS. The compose examples provide OpenAI defaults for the URL, model, and dimensions (https://api.openai.com/v1, text-embedding-3-small, 1536), so the minimal local setup only needs you to fill in AI_API_KEY. Override those values together when using a different embedding provider or model.

Verifier calls send temperature: 0 by default. Set AI_VERIFIER_DISABLE_TEMPERATURE=true to omit the field for providers or models that reject temperature.

Fully Local Setup (Ollama)

Dense-Mem also runs with no hosted AI provider at all. Any OpenAI-compatible endpoint can serve embeddings and verification; with Ollama on the Docker host, pull the two models once and point .env at them:

ollama pull nomic-embed-text
ollama pull llama3.1:8b

AI_API_URL=http://host.docker.internal:11434/v1
AI_API_KEY=ollama
AI_API_EMBEDDING_MODEL=nomic-embed-text
AI_API_EMBEDDING_DIMENSIONS=768
AI_VERIFIER_MODEL=llama3.1:8b
AI_VERIFIER_TIMEOUT_SECONDS=300

Three details matter on this path:

• Use host.docker.internal, not 127.0.0.1; the server calls the provider from inside the compose network. The base compose example maps that name to the Docker host so it also works on Linux, where Docker does not define it by default.

• AI_API_KEY must be non-empty even though Ollama ignores it; startup validation requires a complete embedding configuration.

• Change AI_VERIFIER_MODEL together with the embedding settings. The default is gpt-4o-mini, which does not exist on Ollama. Startup still succeeds; the failure only appears later, at claim-verification time. Pick a model that answers within the verifier timeout on your hardware. A 7B-8B class model verifies comfortably on a laptop; larger models can exceed the default 60-second timeout while they load, leaving claims parked as candidate_claim with the error recorded in the placement item.

Your First Memory

remember stores evidence and returns an ingest_id immediately; placement happens asynchronously. Poll get_memory_placement with that ingest_id to see where the memory landed:

Server-side claim extraction is deliberately conservative: simple first-person statements such as "I prefer ...", "I like ...", or "I use ..." are the reliable way to see the full evidence-to-fact path on a first run. Other phrasings, including third-person forms like "Josh prefers ...", are kept as fragment_only evidence, which recall still returns and ranks below facts. fragment_only is a normal outcome, not an error.

Telemetry Overlay

Prometheus telemetry is optional and off by default. To collect usage, performance, verifier token, embedding token, recall, and promotion metrics for the /ui app and control portal dashboards, run the base stack with the telemetry overlay:

curl -fsSLo prometheus.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/prometheus.yml
curl -fsSLo docker-compose.telemetry.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/docker-compose.telemetry.yml

export TELEMETRY_SCRAPE_TOKEN="$(openssl rand -hex 32)"
docker compose -f docker-compose.yml -f docker-compose.telemetry.yml up -d

The overlay starts Prometheus on 127.0.0.1:9090, retains 30 days of samples, passes TELEMETRY_SCRAPE_TOKEN to Prometheus as a scrape secret, and points Dense-Mem at http://prometheus:9090 for telemetry queries. It also sets TELEMETRY_PROMETHEUS_JOB=dense-mem so dashboards query only the dense-mem scrape job when Prometheus is shared.

Online recall-quality cards use densemem_recall_feedback_total and densemem_recall_feedback_quality_score. They stay at zero until recall feedback is enabled from the control portal config panel and a host LLM submits feedback for recall_memory results. Feedback only omits feedback_comment when quality is high with no negative flags; medium, low, or flagged feedback includes a bounded comment and can include irrelevant result refs for offline analysis. Prometheus still receives only bounded labels; the free-text comment stays in the recall feedback investigation records. When related dreams are returned, feedback can also include bounded dream_feedback judgments without promoting or rejecting the dream automatically. Confirmed true or false dreams should be resolved through resolve_dream_feedback, which records dream-specific telemetry and routes the confirmation evidence through normal memory placement. Normal production recall traffic still contributes request volume, result count, and latency.

For the disposable demo image, keep the control portal disabled and use the demo telemetry overlay instead:

curl -fsSLo prometheus.demo.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/prometheus.demo.yml
curl -fsSLo docker-compose.demo.telemetry.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/docker-compose.demo.telemetry.yml

export TELEMETRY_SCRAPE_TOKEN="$(openssl rand -hex 32)"
docker compose -f docker-compose.yml -f docker-compose.demo.telemetry.yml up -d

The demo overlay scrapes the demo service at demo:8091 on the private Compose network and sets TELEMETRY_PROMETHEUS_JOB=dense-mem-demo. Do not publish that metrics listener publicly.

Compare

Redis is optional for single-node deployments and required for multi-instance deployments.

Documentation

The README is the product overview. The full user documentation lives in the Dense-Mem wiki:

Responsibility Boundary

Dense-Mem is not an agent brain, planner, or external truth arbiter. It stores memory, applies explicit gates, and returns structured outcomes.

Memory Workflow

Direct client tools for claim/fact promotion, raw fragment mutation, raw keyword/vector/graph search, community detection, and retractions are not part of the public client surface. Dense-Mem keeps the underlying logic server-side for verifier, recall, migration, and maintenance flows.

The older *_skill_pack* tool names remain accepted as hidden compatibility aliases, but new clients should use *_memory_pack*. Dense-Mem also exposes MCP prompts through prompts/list and prompts/get; the first bundled prompt, export_memory_as_agent_skill, guides an LLM to recall Dense-Mem experience and draft a self-contained, shareable Agent Skill SKILL.md file for recipients without access to the source memory instance, and without relying on memory-pack import/export.

Memory moves through this path:

remember evidence -> verifier placement -> typed claim -> verification -> promotion gate -> active fact
                         |                                                 |
                         v                                                 v
                  fragment-only / reject                              clarification task

Comparable conflicts are not resolved silently. Dense-Mem returns clarifications[], and the host LLM asks the user which memory is correct. After the user answers, the host calls confirm_memory.

Data Egress

Dense-Mem forwards fragment text and recall queries to the configured embedding provider. Claim verification can send candidate claims and supporting evidence to the configured verifier provider. Self-hosted providers keep that traffic inside your boundary; hosted providers do not. See the wiki Configuration and Technical Reference for provider settings and egress details.

Embedding Model Consistency

Dense-Mem owns embeddings for normal writes and recall. It checks the stored embedding model and dimension on startup so vectors from incompatible models are not mixed silently. Rotation requires re-embedding or rebuilding vector indexes; the step-by-step process belongs in the wiki Configuration.

Tool Discoverability

Dense-Mem exposes three discoverability surfaces backed by one registry:

The full route list and client examples live in the wiki Technical Reference and Quick Start.

Design Notes

• standalone MCP memory architecture

• knowledge-pipeline contracts

• knowledge-pipeline client contracts

• knowledge-pipeline operability

License

Apache-2.0