brain-v42

Persistent memory for coding agents, served over MCP.

brain-v42 gives Claude Code, Codex and any other MCP client a durable second brain: decisions, learnings, code snippets, runbooks, ADRs, tickets and project roadmaps — stored in PostgreSQL, retrieved by full-text + semantic search with reranking, and consolidated every night by an agent pipeline.

• Typed knowledge, not a notes dump — a decision records its WHY and alternatives; a snippet records its intent; a runbook records executable steps. Each type has its own lifecycle (supersession chains, ADR acceptance, learning validation).

• Explicit session lifecycle — the user owns every session boundary. Sessions capture the artifacts they produced, and closing is fail-closed: a session ends with either captured knowledge or an explicit "nothing to capture" reason, never silence.

• Search that ranks — pgvector semantic search + PostgreSQL FTS, fused and re-ranked by a cross-encoder.

• Nightly consolidation ("dream") — an agent pipeline cleans orphan links, merges duplicates, synthesises learnings and proposes promotions, behind per-phase killswitches that all ship closed.

• Multi-project — per-project focus with compare-and-swap revisions, roadmaps, cross-project tickets.

• Observable delivery — versioned delivery contracts bind addressed work to a pull request, persisted CI evidence, integration, and policy-governed fulfillment.

• Measured facts, not remembered ones — a closed catalogue of named probes reads live state (schema head, running release, declared killswitches) against a source identity the operator declared independently, so a briefing states what is rather than what someone last wrote down.

Architecture

Claude Code / Codex (MCP client)
       │ HTTP loopback :8765/mcp (production) · stdio (dev/fallback)
  brain-v42 (FastMCP)
       ├── SQLAlchemy async ─▶ PostgreSQL 16 + pgvector   (source of truth)
       ├── HTTP ─────────────▶ embedding endpoint :8003   (optional, pluggable)
       ├── HTTP ─────────────▶ :8003/rerank               (optional reranker)
       └── bolt ─────────────▶ Neo4j 5 Community          (relationship index, optional)

MCP transport: production = HTTP loopback http://127.0.0.1:8765/mcp; configuration default and dev/fallback = stdio.

PostgreSQL is the single source of truth. Neo4j is a disposable projection fed by a relational ledger/outbox — it can always be rebuilt from PostgreSQL, never the other way around. The canonical path is active in production since 22 July 2026; design and evidence live in docs/ARCHITECTURE.md and the graph ledger runbook.

Embeddings are optional and pluggable, and they degrade gracefully when the endpoint is away — brain_search falls back to full-text search, writes persist with a NULL embedding and are backfilled later. An install with no embedding endpoint at all works.

Two wire formats ship, selected by BRAIN_EMBEDDING_BACKEND:

So a machine without a GPU needs no bundled stack. Point it at whatever serves embeddings, for example a local Ollama:

BRAIN_EMBEDDING_BACKEND=openai
BRAIN_EMBEDDING_SERVICE_URL=http://localhost:11434
BRAIN_EMBEDDING_MODEL=nomic-embed-text
EMBEDDING_DIMENSION=768   # unprefixed on purpose — see Dimension below

Reranking is separately pluggable via BRAIN_RERANK_BACKEND (shim, or cohere for the POST /v1/rerank shape implemented by TEI, Jina and vLLM), and stays best-effort: an unavailable reranker falls back to RRF ordering rather than failing a search.

Instruction prefixes

Asymmetric models (Qodo, E5, BGE) expect queries and documents to be marked differently. Both prefixes default to empty, which is correct for symmetric models and reproduces the unprefixed behaviour exactly:

BRAIN_EMBEDDING_QUERY_PREFIX="query: "        # applied to searches only
BRAIN_EMBEDDING_DOCUMENT_PREFIX="passage: "   # applied to everything written

Keep the trailing space if the model's card shows one — it is part of the prefix. Changing the query prefix is free. Changing the document prefix on a populated corpus requires a full scripts/regen_embeddings.py pass, or the column ends up holding two incompatible vector populations with nothing to flag it.

Dimension

EMBEDDING_DIMENSION is chosen at install time and must be ≤ 2000, the ceiling pgvector's HNSW index accepts. Switching models later means re-embedding the corpus (scripts/regen_embeddings.py).

Set it unprefixed. Every other setting here also answers to a BRAIN_-prefixed name, but the ORM column widths are read straight from EMBEDDING_DIMENSION in db/tables.py, so BRAIN_EMBEDDING_DIMENSION=768 alone would leave the tables at 1536 while the rest of the process believed 768.

One caveat to know before a non-default install: the ORM honours this setting, but four migrations (002, 005, 009, 014) hardcode vector(1536), so a fresh alembic upgrade head creates 1536-wide columns whatever the setting says. Until that is fixed, a non-1536 install needs the columns retyped and their HNSW indexes rebuilt by hand after migrating. If you are running the reference stack at 1536, this does not affect you.

Related MCP server: Muninn

Quick start

git clone https://github.com/hawkixs/brain-v42 && cd brain-v42
uv sync --extra dev --python 3.12      # creates .venv; see "Development" for why not pip
source .venv/bin/activate

# 1. Local Neo4j secret (skip if you run without the graph)
install -d -m 0700 .secrets
read -rsp "Neo4j password (same value as NEO4J_PASSWORD in .env): " PW
(umask 0022; printf 'neo4j/%s\n' "$PW" > .secrets/neo4j-auth); unset PW

# 2. Databases (PostgreSQL 16 + pgvector, Neo4j)
docker compose up -d

# 3. Migrations
export POSTGRES_URL="postgresql+asyncpg://brain:REPLACE_WITH_PASSWORD@localhost:5433/brain"
BRAIN_ALEMBIC_ALLOW_PROD=1 alembic upgrade head

# 4. Run the MCP server (stdio)
python -m brain_v42.mcp.server

Wire it into Claude Code — .mcp.json at the repo root already targets the production HTTP loopback endpoint; for a plain stdio dev setup:

claude mcp add brain-v42 -- python -m brain_v42.mcp.server

BRAIN_ALEMBIC_ALLOW_PROD is required only when the database name is exactly brain; keep it a one-command opt-in, never exported persistently. Alembic rejects DSN query parameters; use the plain form above with host, port, username and password all present.

MCP tools

Full catalog with signatures: docs/MCP_TOOLS.md.

The default catalog profile is compact: the seven session lifecycle tools stay visible, and every other tool is reached through two gateways — brain_find_tool to discover, brain_call_tool to invoke. Set BRAIN_MCP_PROFILE=native to expose every tool directly.

Observable delivery

One delivery path is contract → PR binding → persisted PR/CI observation → integration receipt → explicit requester acceptance. A merge receipt proves the contracted revision reached its target branch. Contracts with acceptance_mode=explicit require the requester to accept the current attempt and delivery digest; automatic contracts can produce fulfillment without that decision.

Brain stores and evaluates this evidence. It does not launch agents, choose work, push commits, merge pull requests, or deploy releases. External orchestrators keep those responsibilities. Delivery reads use persisted observations and make no GitHub calls. With BRAIN_DELIVERY_ENABLED=false, mutations pause while reads and the completion guard for existing contracts remain active.

Ledger and policy: the boundary with red-rail

Brain is the ledger; red-rail is policy. The split is deliberate and it is the reason attestations exist as a separate table (ticket 04bc1f4a).

brain_delivery_attest records one issuer-declared fact about a ticket's delivery workflow — released, deployed, rolled_back, incident_detected, gate_passed and their kin. Brain validates the form and nothing else: the kind must match ^[a-z][a-z0-9_]{0,63}$, the payload must be a bounded JSON object, and the server — never the issuer — computes the digest. Brain never judges what a kind means, never derives a completion refusal from an attestation, and never updates or deletes one. The well-known kinds above are documentation, not an allowlist.

That restraint is what makes the rows usable: red-rail reads them to compute DORA metrics under its own policy, which can change without a schema migration here. Attestations also arrive legitimately after a ticket has closed — an incident or a rollback does not wait for a workflow state — so no ticket-status restriction applies.

For consumers that must not import brain_v42, the whole API is published as data in docs/contracts/delivery_attestations.json — kinds, bounds, the digest recipe with its test vectors, the stable error codes and the list scopes — and a unit test keeps that file equal to the code.

The observer is a separate process with separate credentials. Its dedicated ~/.config/brain-v42/delivery-observer.env must be an owned, regular, non-symlink file with mode 0600. It carries BRAIN_DELIVERY_ENABLED=true, an explicit BRAIN_DELIVERY_POSTGRES_URL, the repository registry, and either a dedicated GitHub token or a complete GitHub App credential set. Keep the observer's GitHub credentials out of the shared application environment; the application keeps its own existing PostgreSQL configuration. The immutable service command is:

<release>/venv/bin/python -m brain_v42.delivery_observer --env-file ~/.config/brain-v42/delivery-observer.env

Each host release lives under ~/.local/share/brain-v42/releases/<full-source-sha>/ and retains the same-SHA source archive, wheel, lock, copied Python 3.12 environment, and hashed manifest. The archive supplies Dream and root scripts that the wheel does not install. Build and installed-wheel checks do not attest a rollout. Follow the immutable delivery release and canary runbook for preflight, activation, evidence capture, and compatible forward rollback.

Sessions

The user controls every session boundary: start, resume, end and abandon are explicit commands, never inferred by a hook, an agent or a client. Sessions capture the durable artifacts they produced into an exclusive ledger, and closing is fail-closed: captured knowledge or an explicit "nothing to capture" reason, never silence.

After 24 hours without a heartbeat, an open session exposes is_stale=true; the marker is derived, the persistent status stays open, and only the 7-day server-side sweep ever abandons a session without an explicit user command.

The full lifecycle contract (capture rules, focus semantics, briefing) lives in docs/MCP_TOOLS.md; the contract is v4 and still evolving.

Configuration (.env)

# Required
POSTGRES_URL=postgresql+asyncpg://brain:REPLACE_WITH_PASSWORD@localhost:5433/brain

# Optional — semantic search and reranking
EMBEDDING_SERVICE_URL=http://localhost:8003
EMBEDDING_DIMENSION=1536              # <= 2000 (pgvector HNSW ceiling)
RERANKER_URL=http://localhost:8003

# Optional — point at any OpenAI-compatible endpoint instead of the bundled stack
BRAIN_EMBEDDING_BACKEND=shim          # shim (default) or openai
BRAIN_EMBEDDING_MODEL=qodo            # model name sent by the openai backend
BRAIN_EMBEDDING_QUERY_PREFIX=         # e.g. "query: " for asymmetric models
BRAIN_EMBEDDING_DOCUMENT_PREFIX=      # changing this needs a full re-embed
BRAIN_RERANK_BACKEND=shim             # shim (default) or cohere

# Optional — relationship graph (safe defaults for a fresh environment)
GRAPH_ENABLED=false
GRAPH_LEDGER_WRITE_ENABLED=false

# Tool catalog profile
BRAIN_MCP_PROFILE=compact   # compact (default) or native

# Observable delivery MCP mutations; reads and existing completion guards remain active
BRAIN_DELIVERY_ENABLED=false

LOG_LEVEL=INFO

Never place MCP_HTTP_TOKEN or MCP_HTTP_DREAM_TOKENS in the shared .env: bearer tokens live in a private 0600 file (~/.config/brain-v42/mcp-token.env), and the graph projector credential in its own (~/.config/brain-v42/graph-projector.env). BRAIN_EMBEDDING_API_KEY and BRAIN_RERANK_API_KEY follow the same rule. Note that pointing either endpoint at a hosted provider sends the text being embedded off this machine — the rest of this deployment is loopback-bound, that step is not. Full reference — every variable, the private secret files, preflights and rollout gates: docs/OPERATIONS.md.

Network trust model

The deployment targets personal agents on a trusted LAN. MCP, PostgreSQL and Neo4j bind to loopback; metrics and automation default to loopback.

Embedding topology: production/default = local unified endpoint http://localhost:8003; deploy/dev-pc is a superseded rollback/reference path.

The reranker shares the unified embedding endpoint :8003/rerank. Treat :8003 as LAN-exposed until you have proved the live bind yourself, and never expose it — or the MCP port — to the Internet. Repository code alone does not prove a live firewall state.

Dream mode

Nightly agent pipeline (scripts/dream.sh: scan → clean → connect → synth → promote → reorg) plus server-side ticket-extraction, roadmap-curation and session-sweep jobs. Every mutating phase sits behind a killswitch and every killswitch ships closed; dry-run is the shipped default. Each phase runs under an exact MCP tool allowlist, and each phase holds a capability bearer scoped to its (project, phase) pair, so a phase sees only the project it was started for.

Phases run on an ordered chain of agent providers (BRAIN_DREAM_AGENT_PROVIDERS), each preflighted before the night starts. When a link is exhausted or unreachable the run falls through to the next rather than failing the phase, and a link that dies before any Brain call is replayable — it is retired for the night and not charged to the retry budget, because a provider that never reached Brain performed no work to redo. Every phase writes which link served it, and what it fell through, to logs/dream/<date>_<project>_<phase>.chain.json. Read that file rather than assuming the first link ran: a night can be entirely green on one provider and prove nothing about the fallthrough. Details: docs/ARCHITECTURE.md and docs/OPERATIONS.md.

Production state

The repository migration target is migration 056. No page in this repository proves a live schema head — measure it, do not read it here:

docker exec brain_v42_postgres psql -U brain -d brain -Atc "select version_num from alembic_version;"

The running build names itself: GET /health returns version (the installed distribution) and alembic_head (the revision shipped with it), both measured, never written by hand.

Measured facts

The sentence above — measure it, do not read it here — is the rule. The facts registry is the mechanism that enforces it, so the session briefing can state the live schema head, the running release and the declared Dream killswitches without anyone retyping them into a document.

A fact is a named, versioned reader. Each one declares its target, its TTL, its timeout, its policies and the exact shape of the value it returns, and the catalogue is closed: probes are registered once at composition and then frozen, so no runtime caller can install a reader of its own.

What makes a reading trustworthy is not the probe but the source identity, and the identity a probe is checked against is declared independently by the operator — never derived from the connection the probe uses. A PostgreSQL reading must match a cluster system identifier, database, address and port the operator wrote down; a live_release reading must match the release SHA and package version the release tooling rendered; a host reading must match a declared hostname. Undeclared means no fact: the probe is refused at registration and the briefing says which one is missing and why, rather than leaving a silent gap. This closes the obvious hole — a probe that reports its own DSN back to you proves nothing about which database it reached.

A measurement has exactly two shapes and no third: Measured, carrying a bounded canonical JSON value and a digest over it, or Unreadable, carrying a closed error_code. A timeout, an unexpected identity or an over-large value each produce an Unreadable that renders as such — never a stale value dressed up as current.

Three targets ship today (production, live_release, host) and the catalogue is declared in src/brain_v42/facts/composition.py. Read it with brain_fact_list and brain_fact_get; facts declared briefing=true also render as lines in the session briefing. Design: docs/superpowers/specs/2026-09-19-measured-facts-and-claims-design.md.

Development

pytest tests/unit -v                          # no PostgreSQL required
pytest --cov=brain_v42 --cov-report=term-missing
ruff check src/ tests/ && ruff format --check src/ tests/
mypy src/

• Stack: Python 3.12+, FastMCP 3.x, SQLAlchemy 2.0 async + asyncpg, Alembic, Pydantic 2, structlog.

• TDD is mandatory — red, green, refactor; tests are never edited to make code pass.

• Coverage floor: 60% (CI blocks below).

• Install with uv sync --extra dev --python 3.12, not with pip. pip install -e ".[dev]" fails on this layout and always has: headless-agents is a uv workspace member ([tool.uv.workspace] + [tool.uv.sources] in pyproject.toml), not a published distribution, so pip looks for it on PyPI and stops with No matching distribution found for headless-agents. The dev toolchain is pinned exactly in uv.lock, so a synced environment resolves to the versions CI runs.

• Pin --python 3.12 explicitly. requires-python is >=3.12, so a bare uv sync on a fresh clone picks the newest interpreter it can find — measured 3.14 — while every CI job, the release job and [tool.mypy] target 3.12. Matching CI is the whole point of the lock; an unpinned interpreter quietly gives it up.

Project layout

brain-v42/
├── src/brain_v42/
│   ├── config.py              # pydantic-settings — single config surface
│   ├── db/                    # SQLAlchemy engine + tables
│   ├── models/                # Pydantic models
│   ├── repositories/          # CRUD + FTS + pgvector + graph adapters
│   ├── services/              # business logic, embedding, reranker, dream, dedup
│   ├── metrics/               # sidecar + collector + cockpit endpoint
│   ├── automation/            # independent webhook/dedup runtime (:9201)
│   └── mcp/                   # FastMCP server + brain_*/dream_* tool handlers
├── tests/{unit,integration}
├── alembic/versions/          # migrations (shipped inside the wheel)
├── scripts/                   # operational CLIs (dream.sh, canaries, repair)
├── services/                  # GPU embedding service + shim + supervisor
├── deploy/                    # systemd units, per-host compose, install.sh
└── docs/                      # ARCHITECTURE, SCHEMA, MCP_TOOLS, OPERATIONS, runbooks

The top-level module graph is enforced acyclic in CI (scripts/check_module_layering.py): any module can still be extracted into a standalone service without dragging a cycle with it.

CI/CD

Stages: lint → test → security → build. Security gates: pip-audit, bandit, gitleaks, container-image pin checks. Docker images are built and pushed on main; there is no deploy stage — rollout to a host is always a manual, out-of-band step. Releases are tag-driven: the release rail builds the wheel + sdist, proves the wheel ships its migrations, and attaches both to the GitHub release.

Versioning

• The shipped version is 0.6.0, and it stays 0.x on purpose: a 1.0.0 would promise a stable interface and a way back, and this project has neither yet.

• No lossless downgrade is promised, at any version. Several migrations protect stored history: 037 refuses when a session capture would be lost, 039 requires an explicit operator opt-in, 053 refuses once delivery workflow history exists, and 054 refuses once a delivery attestation exists.

• Follow the release's operator runbook for recovery. For 0.6.0 (as for 0.5.0), use the compatible forward rollback and keep the repository's migration target in place — 054 since revision 054 shipped. Rollback means selecting a release that supports that head or deploying a forward fix; it never means alembic downgrade, and never means restoring an older dump over a live database.

• 0.6.0 ships the headless-agents workspace member (packages/headless-agents/, version 0.1.0) as a second distribution that brain_v42 depends on; its own version moves independently of this one.

License

Source code: Apache-2.0.

Model weights are not covered by that license, and this is not a formality. The production embedding model, Qodo/Qodo-Embed-1-1.5B, is published under QodoAI-Open-RAIL-M — a license carrying use-based restrictions, not a permissive one. No weights are stored in or distributed by this repository: every model is downloaded from its upstream host at build time, by the operator, who accepts each model's terms directly from its publisher. See NOTICE before redistributing anything.