HAM - Shared Memory for Collaborating Agents

HAM is a PostgreSQL-backed memory service for agents working across sessions, repositories, projects, and tools. A compact local BGE model provides useful semantic recall without an embedding API or GPU; deterministic Clifford and spectral encoders add independent retrieval routes.

Agent connection and collaboration guidance is available in llms.txt. The deployed root URL provides a human-friendly onboarding page with copyable, credential-free agent instructions, while /llms.txt exposes the same setup contract directly to agents.

The production model is deliberately simple and incrementally searchable:

Codex / Claude / Cursor / other MCP clients
  -> local stdio MCP bridge
  -> authenticated HAM HTTP API
  -> local query/passage encoding
  -> PostgreSQL + pgvector (canonical memory, provenance, and derived vectors)

Each deployment can contain many tenants. Within a tenant, agents share memory through explicit scopes while preserving the originating agent, project, repo, task, thread, state, and version of every item.

Agent Workflows

The MCP bridge exposes:

• ham_remember - store a scoped observation, fact, decision, preference, or note.

• ham_recall - semantic and lexical search in the active collaboration context.

• ham_recall_deep - semantic search followed by bounded, scoped cue traversal.

• ham_recent - see recent work without having to invent a search query.

• ham_changes - catch up from an ISO-8601 cursor.

• ham_handoff - publish completed work, next steps, blockers, and touched files.

• ham_get - fetch a complete memory with provenance and version.

• ham_supersede - replace stale knowledge with optimistic concurrency.

• ham_retract - mark incorrect knowledge inactive without deleting its history.

• ham_reflect and ham_context - close and reopen working sessions.

Scopes are arbitrary lower-case labels such as shared, project:ham, repo:monumentalsystems/ham, or task:deploy. MCP clients should use the same tenant ID and distinct agent IDs.

Related MCP server: Memclaw

Quick Start

Create .env from .env.example, generate strong values for HAM_DB_PASS and HAM_API_KEY, then run:

docker compose up --build -d
curl http://127.0.0.1:8042/health/ready

The Compose stack starts a private pgvector/pgvector:pg16 database and binds the API to 127.0.0.1:8042. The entrypoint creates the required vector, pg_trgm, and intarray extensions and applies all idempotent migrations in a serialized transaction. The default 384-dimensional BGE-small model is baked into the API image. Each write is embedded immediately, while startup performs a background batch backfill for rows created before semantic retrieval was enabled. Each batch commits independently, readiness does not wait for the corpus, and normal operation has no separate global reindex step. Set HAM_SEMANTIC_BACKFILL_ON_START=false to run without automatic backfill and HAM_SEMANTIC_BACKFILL_BATCH_SIZE to control transaction size.

The default model is BAAI/bge-small-en-v1.5, loaded through FastEmbed's quantized ONNX runtime. HAM_SEMANTIC_MODEL may be changed only to a 384-dimensional FastEmbed text model; the stored model name keeps vectors from incompatible encoder versions out of the same search.

To test an authenticated request:

curl -H "Authorization: Bearer $HAM_API_KEY" \
  -H "X-GB-User-ID: shared-team" \
  http://127.0.0.1:8042/stats

MCP Configuration

The stdio process is a thin bridge to the shared API. In normal shared-service mode it requires HAM_API_URL and does not need database credentials. Before cloning, check likely workspace directories for an existing checkout containing pg_ham/mcp_server.py, and test the exact Python interpreter used by the client.

{
  "mcpServers": {
    "ham": {
      "command": "python",
      "args": ["-m", "pg_ham.mcp_server"],
      "cwd": "/path/to/ham",
      "env": {
        "HAM_API_URL": "http://127.0.0.1:8042",
        "HAM_MCP_USER_ID": "shared-team",
        "HAM_AGENT_ID": "codex",
        "HAM_SCOPES": "shared,project:ham,repo:monumentalsystems/ham",
        "HAM_PROJECT": "ham",
        "HAM_REPO": "MonumentalSystems/ham"
      }
    }
  }
}

Give each agent a distinct key in HAM_AGENT_KEYS_JSON, and expose that key as HAM_API_KEY in the environment that starts the client. Do not place a live key in MCP configuration, a repository file, shell history, or an agent prompt. Codex can whitelist the inherited variable with env_vars = ["HAM_API_KEY"]. Clients that cannot safely inherit it can run scripts/ham_mcp_launcher.py with HAM_CREDENTIAL_COMMAND_JSON set to a non-interactive secret-manager command. The command is executed without a shell and must print one credential line. See llms.txt for Claude and Codex examples.

The server binds the key to its tenant and agent identity; client-supplied identity headers cannot override it. Set HAM_TASK and HAM_THREAD when the host can provide them. Local clients can reach a private remote deployment through an SSH tunnel without exposing HAM or PostgreSQL.

Restart the MCP client after adding or changing the server; tools are discovered at client startup. Verify with ham_stats, ham_recent, and ham_context.

For an outside collaborator, add allowed_scopes to the credential. The server then defaults all reads and writes to that allowlist, rejects explicit broader scope requests, hides out-of-scope memory IDs and aggregate statistics, and allows lifecycle changes only to memories created by that credential's agent. This permits project sharing inside an existing tenant without exposing or mutating unrelated tenant memory:

{
  "key": "generate-a-collaborator-secret",
  "tenant_id": "shared-workspace",
  "agent_id": "external-alice",
  "role": "agent",
  "allowed_scopes": ["project:alpha", "repo:org/alpha"]
}

Scopes are the authorization boundary; project, repo, task, and thread remain provenance and query filters. Memories intended for the collaborator must include at least one of the credential's allowed scopes.

Legacy direct-PostgreSQL MCP mode is disabled by default. Local research setups can opt in with HAM_MCP_DIRECT_DB_ENABLED=true and the HAM_DB_* variables, but this grants the MCP process database credentials and is not recommended for shared or multi-user deployments.

Collaboration Contract

Per-agent keys are the normal tenant boundary. HAM_API_KEY is the separate administrator key and may select a tenant with X-GB-User-ID; it is required for hard delete, reset, graph rebuild, enrichment, and research endpoints. Unsigned identity headers are accepted only when HAM_TRUST_IDENTITY_HEADER=true behind an authenticated proxy. HAM_SINGLE_USER_ID is for local development.

Every collaborative memory stores standardized JSON metadata:

Rows also carry state, version, timestamp, created_at, updated_at, and supersedes_id. timestamp is event time, created_at is immutable ingestion time, and updated_at advances on lifecycle changes. ham_recent orders active memories by created_at; the change feed includes superseded and retracted items ordered by updated_at so agents can update their local understanding.

Durability is an expected-staleness signal, not a relevance score:

When a hot tier exceeds capacity, HAM cools memories with the greatest elapsed half-lives first, offset by access. It never deletes them, and durability does not change query-time semantic relevance. Memory types provide useful defaults; agents may set durability explicitly when the expected lifetime differs.

API Surface

Legacy and research retrieval endpoints remain available. PostgreSQL is the production authority. Set HAM_SEMANTIC_ENABLED=false only for a deliberate harmonic/lexical fallback. QKPS and legacy pickle-backed fields are disabled by default (HAM_QKPS_ENABLED=false, HAM_LEGACY_FIELD_ENABLED=false) because they are process-local accelerators and should not define shared consistency.

Request Tracing

Send X-HAM-Trace-Level: summary to receive X-HAM-Trace-ID and Server-Timing response headers and emit one structured server trace event. An optional valid X-HAM-Trace-ID is propagated end to end. The MCP bridge does this automatically and logs client latency, first-request state, and server timing to stderr when HAM_MCP_TRACE_ENABLED=true (the default). Trace events contain operation names, timing, status, and result counts; they exclude memory content, queries, credentials, tenant IDs, and agent IDs.

Security

• Keep PostgreSQL private and give agents only HAM_API_URL and HAM_API_KEY.

• Prefer a private network or SSH tunnel; do not publish HAM directly to browsers.

• Leave CORS empty unless a trusted browser application genuinely needs it.

• Use per-agent credentials for every MCP client. Keep the administrator key out of agent environments; it can hard-delete or reset tenant data.

• Bind outside collaborators to explicit allowed_scopes; never rely on their MCP defaults as the security boundary.

• Application authorization is enforced before queries. PostgreSQL row-level security remains useful future defense in depth for mutually untrusted operators.

• Back up PostgreSQL off-host and test restoration.

Development

pip install -r requirements.txt -r requirements-dev.txt
python -m py_compile pg_ham/server.py pg_ham/mcp_server.py scripts/e2e_collaboration.py scripts/ham_mcp_launcher.py
pytest -q
ruff check pg_ham/server.py pg_ham/mcp_server.py scripts tests

Training and benchmark scripts remain research tooling. Generated datasets, checkpoints, embeddings, field snapshots, and logs must stay out of git. scripts/benchmark_semantic_retrieval.py is an encoder smoke test over a small curated corpus. It does not measure fused PostgreSQL ranking, cue traversal, multi-hop recall, or temporal ordering and must not be treated as a system quality benchmark.

License

HAM is available under the Apache License 2.0.