MCP Shared Memory Server

A PostgreSQL-backed memory service that lets multiple MCP clients share, revise, and search a project's knowledge without ever resurfacing a decision that has been replaced.

Claude Desktop, Cursor, and Claude Code each keep their own context today. A decision explained to one is invisible to the others. Every new session starts from zero.

This project is a single MCP server, backed by one PostgreSQL database. It gives all of them one shared, versioned memory — conflict-safe writes, a full revision history, and hybrid keyword-plus-vector search. It also guarantees, structurally, that a replaced decision can never resurface as a search result.

It needs no separate vector database, no Redis, and no Kafka. Just PostgreSQL — which a backend project already needs anyway.

The problem

Developers don't use just one AI coding tool. Someone explains a decision to Claude Desktop. Later, they switch to Cursor for a different task. Cursor has no idea what was just decided — it never saw that conversation. Every tool starts from zero, every time.

The fix sounds simple: give every tool one shared place to read and write project knowledge. Giving one AI assistant a memory really is simple — it's just a text file it can read. Sharing that memory correctly across several different clients is where it gets hard. Here's why:

• two clients can try to update the same fact at the same time

• a decision made last month can be reversed, and the reversal has to actually take effect everywhere

• the old decision still needs to be auditable — someone will ask why it changed

• if both the old and new versions stay retrievable, a similarity search can return the wrong one — the retired wording often matches the query better than the current answer does

• lexical and semantic search each miss things the other catches, so neither alone is enough

• whatever gets retrieved has to fit inside a token budget, not just be sorted by relevance

• every client has to reach all of this through the same protocol, not a bespoke integration each

Storing a fact is easy. Correctly retiring one is not. A retired fact must never come back, no matter how it's searched for. It must also stay in the audit trail. That combination is a concurrency and retrieval problem.

Related MCP server: memory-service

What this project solves

Every MCP client reads and writes one shared, versioned memory. If two clients write at the same time, the conflict is resolved safely — nothing is silently overwritten. When a fact is replaced, the old version is marked as replaced in the very same transaction as the new one. The old version stays fully readable in the audit history. But it can never come back as a search result again.

That last guarantee is structural, not a ranking decision. A replaced fact is excluded from retrieval before any scoring runs. So no query, and no similarity match, can bring it back.

• Sending the full 200-memory evaluation corpus as context costs about 7,583 tokens.

• A single memory_context call, at its default 2,000-token budget, returns the 44 most useful memories instead.

• That request costs only 1,797 tokens — about 4.2x fewer tokens.

• Full method in docs/eval/tokens.md.

Architecture

flowchart TD
    C1["Claude Desktop<br/>(own server process)"] -->|stdio / JSON-RPC| S
    C2["Cursor<br/>(own server process)"] -->|stdio / JSON-RPC| S
    C3["any other MCP client"] -->|stdio / JSON-RPC| S

    S["memhub-server<br/>7 MCP tools"] --> SVC["Service layer<br/>CAS revise · dedup · idempotency"]
    SVC --> PG[("PostgreSQL<br/>memories · revisions · outbox")]

    PG --> FTS["Full-text search<br/>tsvector + GIN"]
    PG --> VEC["pgvector<br/>HNSW, cosine distance"]
    FTS --> RRF["Reciprocal Rank Fusion"]
    VEC --> RRF
    RRF --> FILTER["Stage-0 filter<br/>excludes superseded / deleted / expired"]
    FILTER --> BUDGET["Token-budgeted context<br/>quotas + MMR + knapsack fill"]

Each client spawns its own copy of the server, as a subprocess. They share no memory and no cache with each other. PostgreSQL is the only channel between them. That's what makes the concurrency control below a real requirement, not a nice-to-have.

What this server actually sees: only the arguments of the tool calls made to it — nothing more. It never receives conversation transcripts, and it has no access to a client's chat history. This is a shared memory system — clients decide, explicitly, what's worth recording. It is not, and will never be, a system that transparently syncs chat history.

Why vector search alone isn't enough

Embedding a memory, storing the vector, and running a nearest-neighbour search gets you a working prototype fast. But it breaks the moment a memory is revised. Nothing about that approach stops the old embedding from still being the closest match to a new query.

What makes revision safe sits above the vector database, not inside it: conflict-safe writes, immutable revisions, supersession, and a structural filter that excludes retired memories before ranking ever runs. All of that is wrapped in one hybrid retrieval path, and exposed over MCP.

Revision history and supersession

Two distinct mechanisms, both immutable, and worth telling apart:

• memory_revise creates a new revision of the same memory (revision 1 → 2 → 3...). It's guarded by compare-and-set. Use it to correct or extend a fact's wording.

• memory_remember ... supersedes=[...] retires one memory and asserts a new, separate one in its place, in a single transaction. Use it when the decision itself changes.

sequenceDiagram
    participant Cursor
    participant DB as PostgreSQL

    Note over Cursor,DB: Monday — a decision is recorded
    Cursor->>DB: memory_remember(DECISION, "queue runs on Redis")
    DB-->>Cursor: memory A, status=ACTIVE

    Note over Cursor,DB: Six months later — the decision is reversed
    Cursor->>DB: memory_remember(DECISION, "queue runs on PostgreSQL SKIP LOCKED", supersedes=[A])
    DB-->>Cursor: memory B, status=ACTIVE

    Note over DB: One transaction: A → SUPERSEDED, B → ACTIVE

    Cursor->>DB: memory_search("redis")
    DB-->>Cursor: only memory B — A is structurally excluded

    Cursor->>DB: memory_history(A)
    DB-->>Cursor: A, status=SUPERSEDED, superseded_by=B — still fully readable

The exclusion of A is not a ranking decision. A similarity search would happily return it — it literally contains the word "Redis," and the replacement barely mentions it. Instead, it's a filter that every retrieval path runs before ranking ever sees a candidate. tests/integration/test_stale_memory.py checks this at every limit from 1 to 100. And mutation-testing the filter — by removing the status condition on purpose — makes five of those tests fail, confirming it's real.

Concurrent writes

Two clients can read the same revision and try to update it at the same time. One has to win. The other has to be told it lost — not silently overwritten.

sequenceDiagram
    participant A as Client A
    participant B as Client B
    participant DB as PostgreSQL

    A->>DB: read memory, revision = 4
    B->>DB: read memory, revision = 4
    A->>DB: memory_revise(expected_revision=4)
    DB-->>A: OK — now revision 5
    B->>DB: memory_revise(expected_revision=4)
    DB-->>B: outcome="conflict", current_revision=5, current_content=...

memory_revise is a single-statement compare-and-set:

UPDATE memories SET current_revision_no = current_revision_no + 1
 WHERE id = :id AND project_id = :pid AND current_revision_no = :expected AND status = 'ACTIVE'

Zero rows updated means another writer already moved the revision forward. The correctness argument is a PostgreSQL mechanism called EvalPlanQual. When a blocked transaction unblocks, PostgreSQL re-checks this WHERE clause against the newest committed row. So the read and the write happen as one single statement. There's no gap where a second writer could act on stale information.

READ COMMITTED is chosen deliberately here, over SERIALIZABLE. SERIALIZABLE would turn every losing writer's clean, informative refusal into an opaque 40001 retry instead.

tests/concurrency/ proves it directly. 50 writers are launched from a barrier, so they truly collide at the same instant. Exactly 1 succeeds. The other 49 receive a conflict, with the winning revision attached. The invariant suite then confirms the database agrees. Removing the version predicate from the SQL — a one-line change — makes those tests fail. Verified.

A separate mechanism handles the other half of correctness under concurrency. Idempotency is one client retrying the same request, after a dropped connection. It's keyed on a caller-supplied client_request_id. The retry replays the original stored response, instead of writing a second time.

Deduplication is different: two different clients independently asserting the same fact. It's keyed on a normalised content hash. Instead of creating a duplicate, it returns the existing memory and records the second assertion as corroborating evidence.

These are easy to mix up, and they solve different problems. tests/concurrency/test_idempotency.py keeps them distinct.

Retrieval architecture

Full-text search and pgvector similarity both run over the same stage-0 filter. Their two rankings are then combined by Reciprocal Rank Fusion — using position, not score.

Why not just add the scores together? ts_rank_cd is unbounded and depends on the corpus; cosine distance always lives in [0, 2]. Adding those two numbers means nothing. Normalising each one per query is worse — it scales a mediocre best match up to 1.0, exactly like a perfect one.

Approximate nearest-neighbour search always returns the k closest vectors — whether or not anything is actually close. So a cosine-distance threshold (0.35) gates the semantic leg. That number was chosen by sweeping it against the corpus, not by intuition (see docs/eval/threshold-sweep.md).

Without that threshold, hybrid retrieval actually scored a better nDCG (0.881) — but precision collapsed to 0.113. Every unanswerable query started returning ten confident-looking, irrelevant results instead of none.

Retrieval quality — measured

A hand-graded dataset: 200 memories, 34 queries, all written before any retrieval strategy was measured against them. The numbers are gated against a committed baseline. So a regression fails the build, instead of going unnoticed.

The last column is the actual point — not the first three. A retired memory reached a caller zero times. That held at every strategy, every token budget, and every query, including one built specifically to defeat a similarity-only system: searching "redis", when the retired memory is about Redis and the current answer only mentions it to say it was removed.

The jwt query went from 0.000 to a perfect 1.000 under hybrid search. Plain full-text search scores 0.000 because the Snowball stemmer never matches JWTs. Semantic similarity does bridge that gap.

deadlock prevention stays at 0.000 either way. The matching memory describes deadlock prevention without ever using that exact phrase, and 384 dimensions from a small local model isn't enough to close that gap. This is recorded as an open case, not smoothed over.

Consistency guarantees

Precise claims only — no "strong consistency" without saying what that means here.

• A stale write can never overwrite the current revision. Enforced by the compare-and-set above, not by application logic.

• Exactly one current revision per memory, at all times. UNIQUE (memory_id) WHERE is_current — a database constraint, not a convention.

• Superseded, deleted, and expired memories never appear in normal retrieval. One stage-0 predicate enforces this, and every retrieval path runs through it.

• A memory can only be superseded within its own project. A composite foreign key makes cross-project supersession structurally unrepresentable, not merely disallowed.

• Every timestamp comes from the database's clock, never an application clock. So there's no clock-skew window between processes to worry about.

• Content is never destroyed by a normal operation. Revisions are append-only. The one destructive path, memhub-admin purge, is a separate command. It's audited, human-invoked, and deliberately unreachable over MCP.

9 of the 14 invariants listed in docs/architecture.md are enforced at the schema level. That means they hold even if a bug reaches the service layer. tests/integration/test_invariants.py proves each one directly — it bypasses the service layer entirely and attempts the forbidden write straight against the database.

Failure model

Driver-level failures get classified into codes that say what to do next. The distinction between the codes is the point — not how many there are:

UNKNOWN_OUTCOME is the one genuinely ambiguous case. The transaction either committed just before the connection dropped, or it didn't — and the acknowledgement that would have said which one is exactly what got lost.

So the response doesn't claim the write failed. Instead, it names the two ways to actually find out: replay the idempotency key, or re-read the data. This branch order is mutation-tested — inverting it makes three tests fail, because it would report every mid-flight disconnect as safely retryable. That's exactly how duplicate writes happen.

docs/failure-modes.md maps every failure the architecture claims to handle to the specific test that proves it. It also states plainly which ones are arguments rather than tests, and explains why — in those cases, a test would just be testing PostgreSQL itself.

Testing strategy

369 tests, organized by what they're actually checking, not just counted:

A few tests worth naming directly. The 50-writer compare-and-set test only passes for the right reason because a fixture first checks that the connection pool can actually supply 50 distinct backends. Without that check, a 50-way test against a 10-connection pool would just measure five sequential waves — and still pass, for the wrong reason.

The stage-0 filter, and the failure classifier's branch order, are both mutation-tested. Each mechanism was deliberately broken on purpose. The relevant tests were confirmed to fail. Then the code was restored. That's the only real evidence a test was checking anything at all.

Performance and scaling

Measured at three corpus sizes, on a local Docker Desktop PostgreSQL instance. Server time comes from EXPLAIN ANALYZE; client time includes transport overhead on top of that:

  1,000 memories   server  0.36ms   client  5.34ms
 10,000 memories   server  0.53ms   client 11.00ms
100,000 memories   server  0.67ms   client 22.80ms

The corpus grew 100×. Server-side query time only grew 1.9×. The gap between server time and client time is Docker Desktop's port forwarding — not query cost. That's why the benchmark reports both numbers separately, instead of folding the overhead into one misleading number.

At 100,000 rows, the planner still chooses a sequential scan over the GIN index — and it's correct to. With LIMIT 10, the scan stops as soon as it finds ten matches, before an index lookup plus heap fetches would even pay for themselves. This is recorded in docs/perf/scaling_plan.txt rather than asserted on — two earlier versions of this benchmark asserted the wrong thing here.

This measures one selective query, at three corpus sizes, on one machine. It shows sub-linear growth for that specific query shape. It is not a general scalability claim.

MCP protocol and tool surface

MCP gives an AI client a standard way to discover and call tools exposed by a separate process. It's just the transport. The real engineering in this repository is the memory-consistency, revision, retrieval, and concurrency layer sitting behind it. The seven tools below are a thin surface on top of that service layer.

The manifest — names, descriptions, schemas — is snapshotted to tests/protocol/manifest.json, and checked on every run. Tool descriptions are the actual prompt that steers the calling model. So a wording change alone can alter behaviour, with no logic change at all. The snapshot turns that into a visible diff.

One workflow, end to end

1. Claude Desktop remembers: "The job queue runs on Redis."          [session ends]
2. Cursor, a separate process sharing nothing but the database,
   searches "queue" — finds it, with author_client recorded.
3. Months later, Cursor remembers the replacement, superseding #1
   in the same transaction.
4. Search for "redis" now returns only the current answer, even
   though #1 still contains that exact word.
5. memory_history on #1 shows it as SUPERSEDED, not gone.

A runnable version of this is in demo.py — it drives the real stdio transport and prints the actual tool responses.

Quick start

docker compose up -d --wait        # PostgreSQL 16 + pgvector
pip install -e ".[dev]"
alembic upgrade head
pytest -v                          # 369 tests against the real database
python demo.py                     # watch the workflow above run for real

Connecting a real client (Claude Desktop, Cursor) is covered in docs/clients.md.

Repository structure

src/memhub/
  domain/          pure types, policy, validation, normalisation — no I/O
  services/        transactions, invariants, policy — no MCP awareness
  persistence/     ORM models, repositories (every method requires a project scope)
  retrieval/       filters.py, semantic.py, fusion.py — the stage-0 filter, written once
  embeddings/      the port, a local model, and a deterministic fake for CI
  mcp/             thin handlers, schemas, error mapping, stdio entry point
  cli/             operator commands (purge, gc, status) — deliberately not over MCP
  observability/   JSON logs to stderr; in-process metrics registry
migrations/        async Alembic
tests/             unit, integration, concurrency, failure, perf, protocol, eval
docs/architecture.md   the full design, including what was deliberately not built
docs/failure-modes.md  every failure the design claims to handle, mapped to its test

Key design decisions

PostgreSQL as the only source of truth, including as a job queue. Embedding generation is enqueued with FOR UPDATE SKIP LOCKED, in the same transaction as the write it's for. So either both exist, or neither does. No Redis, no Kafka — the durability a job queue needs was already sitting in the database being written to anyway.

Hybrid FTS + pgvector, instead of a separate vector database. Both retrieval paths run inside the same transaction boundary as the stage-0 filter. So "superseded memories are excluded" is one single guarantee — not two separate systems that both have to agree.

Optimistic concurrency (compare-and-set), instead of pessimistic locking. A conflicting writer gets an immediate, informative refusal, with the winning revision attached. It never has to block behind a lock, or receive a generic serialization failure.

Reciprocal Rank Fusion, instead of blending scores. Lexical and vector scores live on incompatible, unbounded scales. Combining rank positions instead means never needing a shared scale at all.

Structural exclusion, instead of ranking-based suppression. A retired memory is removed from the candidate set before any ranking even runs. So no scoring function — today's, or a future one — can ever accidentally bring it back.

Immutable revisions and append-only history. Nothing is ever overwritten. A mistaken decision is recorded as superseded, not erased — because the audit trail is part of the product, not an afterthought.

Tech stack

Language: Python 3.12 Protocol: MCP (mcp SDK v2), stdio transport Persistence: PostgreSQL 16, SQLAlchemy 2.x (async), asyncpg, Alembic Retrieval: PostgreSQL full-text search, pgvector (HNSW), BAAI/bge-small-en-v1.5 via fastembed Testing: pytest, pytest-asyncio, real PostgreSQL throughout — no mocked database Infrastructure: Docker Compose

Project status and limitations

Not built: an OTLP/Prometheus metrics exporter — metrics are collected in-process only, and nothing scrapes or receives them yet. Also not built: MCP resource subscriptions, and a shared-process Streamable HTTP transport, for cases where stdio's one-process-per-client model doesn't fit.

Recommended engineering improvements

Honest gaps, not disguised as finished work:

• No authentication or authorization layer. Any process that can reach the configured MEMHUB_DATABASE_URL can read and write any project. That's fine for the single-user local setup this targets. It becomes a real gap the moment one shared server serves multiple untrusted callers.

• No metrics exporter. The in-process registry correctly enforces label-cardinality discipline. But nothing currently scrapes or receives that data — there's no operational visibility beyond the structured logs.

• stdio is single-tenant per client process. A long-lived server handling many concurrent client connections would need the Streamable HTTP transport instead. That's a real design change — it brings authentication and per-caller authorization with it — not a simple flag to flip.

• Retention is manual. memhub-admin gc collects spent idempotency keys and dead embedding jobs. But nothing schedules it automatically — a human or an external cron has to invoke it; the system never does it on its own.

License

MIT