Dense Mem

Dense-Mem is a standalone HTTP MCP memory server using Streamable HTTP. It stages exact evidence, derives semantic state through validated server policy, and returns active evidence contexts with graph-shaped Relationship handles. PostgreSQL is the durable authority for knowledge, lifecycle, provenance, search, authorization, and audit; Redis is coordination only. A single-node deployment may use process-local coordination; a multi-instance deployment requires Redis or an equivalent distributed coordination implementation.

The host LLM owns conversation and judgment. Dense-Mem owns durable evidence, owner authorization, lifecycle events, support eligibility, and bounded recall. The external memory automation contract is MCP at /mcp; browser routes are first-party interfaces, not an alternative public automation API.

Dense-Mem is part of the research preprint Governed Enterprise AI Memory Beyond RAG: From Vector Retrieval to Permissioned Knowledge Graphs.

Try the Hosted Demo

Create a temporary isolated team at https://demo-dense-mem.markhuang.ai to test disposable data before self-hosting.

Related MCP server: memlawb

Why Dense-Mem

• Evidence is exact, durable, and append-only. A lifecycle action changes its effective state without deleting provenance or trace lineage.

• Entity and typed Value are semantic nodes. Profile-owned Relationships become active graph edges only when their evidence support is eligible.

• Provider output is a proposal. Closed-schema validation and deterministic server policy decide durable state.

• Default recall excludes candidates and Hypotheses and returns evidence only when its active Relationship support path is eligible for the requested time.

• Team visibility and profile mutation authority are distinct. An author can change only their own evidence or owned semantic records.

60-Second Quickstart

Download the local compose example and environment template, configure the required secrets, and start Dense-Mem:

mkdir dense-mem-local
cd dense-mem-local

curl -fsSLo docker-compose.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/docker-compose.base.yml
curl -fsSLo .env.example \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/.env.example

cp .env.example .env
# Fill in POSTGRES_PASSWORD, CONTROL_PORTAL_TOKEN, and AI_API_KEY.
${EDITOR:-vi} .env

docker compose up -d

The base stack uses PostgreSQL with pgvector as the durable authority. Leave NEO4J_* unset for normal operation; a legacy Neo4j corpus is migration input, not a runtime fallback. The local ports are:

MCP:            http://127.0.0.1:8080/mcp
User portal:    http://127.0.0.1:8080/ui
Control portal: http://127.0.0.1:8090/

Open the control portal with CONTROL_PORTAL_TOKEN, then create a team and its first profile/API key. For control-plane automation, use the same private API:

control_token="<CONTROL_PORTAL_TOKEN from .env>"

curl -fsS -X POST http://127.0.0.1:8090/control/api/teams \
  -H "Authorization: Bearer ${control_token}" \
  -H "Content-Type: application/json" \
  -d '{"name":"primary-memory"}'

curl -fsS -X POST http://127.0.0.1:8090/control/api/teams/<team-id>/profiles \
  -H "Authorization: Bearer ${control_token}" \
  -H "Content-Type: application/json" \
  -d '{"name":"default profile"}'

The release image contains one project executable, /app/server. It applies pending PostgreSQL migrations under a database session lock before serving, so the Compose stack does not need a separate migration container. Multiple server replicas that share one writable primary serialize this startup step. Keep rolling-deployment migrations backward compatible with the previous app version; independent databases must each be migrated by a server connected to that database. Administration stays on the private control portal/API, while dreaming and automatic conflict review run as server background workers.

The image healthcheck allows the default 30-minute migration window and becomes active after its first success. If POSTGRES_MIGRATION_TIMEOUT_SECONDS is set above 1800, override the deployment healthcheck start period to at least the same duration.

Release candidates use vX.Y.Z-rc.N and demo-vX.Y.Z-rc.N. Stable releases use vX.Y.Z, latest, and demo-vX.Y.Z; there is no rolling demo tag.

The server requires complete embedding and verifier configuration at startup: AI_API_URL, AI_API_KEY, AI_API_EMBEDDING_MODEL, AI_API_EMBEDDING_DIMENSIONS, and AI_VERIFIER_MODEL. The compose examples provide OpenAI defaults for embeddings; choose the chat models explicitly in .env.

Verifier and assessor calls send temperature: 0 by default. Set AI_VERIFIER_DISABLE_TEMPERATURE=true to omit the field for providers or models that reject temperature.

Fully Local Setup (Ollama)

Any OpenAI-compatible endpoint can provide embeddings and verification. With Ollama running on the Docker host:

ollama pull nomic-embed-text
ollama pull llama3.1:8b

AI_API_URL=http://host.docker.internal:11434/v1
AI_API_KEY=ollama
AI_API_EMBEDDING_MODEL=nomic-embed-text
AI_API_EMBEDDING_DIMENSIONS=768
AI_VERIFIER_MODEL=llama3.1:8b
AI_VERIFIER_TIMEOUT_SECONDS=300

Use host.docker.internal, not 127.0.0.1, because the server calls the provider from the compose network. AI_API_KEY must remain non-empty because startup validation requires a complete provider configuration.

• Set AI_VERIFIER_MODEL to a model that exists on the selected chat endpoint. Startup validates the model configuration before the service accepts memory writes. A 7B-8B class model works for local smoke tests; larger models can exceed the default 60-second timeout while they load, leaving placement attempts retryable until the model responds.

Evidence Lifecycle

remember durably stages exact evidence and returns an ingest_id; provider calls and placement happen after acknowledgement. Poll get_memory_placement for the authoritative processing state.

To replace a specific current evidence item you own, put its UUID in the new item's supersedes_evidence_ids. Direct targeting is separate from advancing a source revision with previous_source_revision; do not combine them.

{
  "evidence": [
    {
      "content": "The deployment target is now PostgreSQL only.",
      "source_type": "manual",
      "supersedes_evidence_ids": ["<owned-current-evidence-uuid>"],
      "idempotency_key": "deployment-target-correction-20260729"
    }
  ]
}

The target is retired atomically when the replacement is accepted for intake, even if later placement is rejected or quarantined. This preserves the exact correction decision instead of silently leaving stale evidence effective.

To retract evidence without a replacement, call retract_evidence with owned current IDs, a bounded reason, and an idempotency key:

{
  "evidence_ids": ["<owned-current-evidence-uuid>"],
  "reason": "The source was withdrawn.",
  "idempotency_key": "withdrawn-source-20260729"
}

Both operations append lifecycle events. They never physically delete evidence or trace lineage. Current recall excludes retired evidence, while a historical known_at view before the event can still show what the system knew then.

Recall and Graph State

recall_memory is evidence-first but support-path gated. Its results[] contain evidence contexts only after final hydration proves an active, query-relevant Relationship support path remains eligible for the requested valid_at and known_at view. Related Relationships, communities, and Hypotheses are separate bounded fields; candidates and Hypotheses are not default memory results.

remember evidence (+ optional Entity/Relationship proposals)
        |
        v
durable staging -> validated placement -> active eligible Relationships
        |                                      |
        +-- lifecycle event -------------------+
                                               |
                                               v
                         support-gated evidence recall and trace lineage

MCP Tool Catalog

The active contract is dense-mem.v2.4. Discover the authorized catalog with MCP tools/list; the server applies the same scope, feature, and visibility checks to tools/call.

Memory-pack writers emit dense-mem.memory-pack.v2.4; strict readers preserve support for prior v2.3 and v1 artifacts after validating their original hashes.

Supported HTTP Surfaces

There is no supported public REST memory API. Do not automate browser routes or depend on retired /api/v1 paths.

Telemetry Overlay

Prometheus telemetry is optional and off by default. To collect HTTP, embedding, verifier, recall, feedback, and conflict-review metrics for the first-party dashboards, start the base stack with the overlay:

curl -fsSLo prometheus.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/prometheus.yml
curl -fsSLo docker-compose.telemetry.yml \
  https://raw.githubusercontent.com/markhuangai/dense-mem/main/examples/docker-compose.telemetry.yml

export TELEMETRY_SCRAPE_TOKEN="$(openssl rand -hex 32)"
docker compose -f docker-compose.yml -f docker-compose.telemetry.yml up -d

The overlay starts Prometheus on 127.0.0.1:9090 and scopes dashboard queries to TELEMETRY_PROMETHEUS_JOB=dense-mem. Free-text recall-feedback comments stay in bounded investigation records; Prometheus receives only bounded labels.

Responsibility Boundary

Data Egress and Consistency

Dense-Mem can send evidence text, proposal context, and recall queries to the configured embedding and verifier providers. Self-hosted providers keep that traffic within your boundary; hosted providers do not. Embeddings are derived, versioned state and cannot overwrite newer sources. Startup checks prevent mixing incompatible embedding models or dimensions.

Documentation

License

Apache-2.0