Skip to main content
Glama
Neuralbrain MCP Server

Neuralbrain MCP Server

by derricke
README.md
# Neuralbrain

A standalone, deterministic memory service — a **"brain"** — for autonomous and interactive coding agents (Google Antigravity, Claude Code, Claude Desktop, Cursor, and custom orchestrators).

Extracted from the NeuralSwarm agent-coding orchestrator and generalised from swarm roles to **projects, notes, and task trajectories**, serving any coding agent over HTTP REST and Model Context Protocol (MCP).

---

## Navigation & Architecture Documentation

| Document | Purpose |
|---|---|
| **[AGENTS.md](AGENTS.md)** / **[CLAUDE.md](CLAUDE.md)** | The Constitution: fundamental rules, architectural invariants, and operational mandates |
| **[DECISIONS.md](DECISIONS.md)** | Architectural Decision Records (ADRs) and active design considerations |
| **[docs/BUILD-ORDER.md](docs/BUILD-ORDER.md)** | 11-step implementation roadmap and verification log |
| **[docs/SCHEMA.md](docs/SCHEMA.md)** | Complete SQLite DDL, indexes, triggers, and Qdrant payload schema |
| **[docs/API.md](docs/API.md)** | Full HTTP REST endpoint specifications, request/response schemas, and status codes |
| **[docs/MCP.md](docs/MCP.md)** | Model Context Protocol tool definitions (`brain_search`, `brain_save`, etc.) |
| **[docs/CONTRACTS.md](docs/CONTRACTS.md)** | Consolidation JSON schemas and 8-check mechanical verification gate |
| **[docs/EMBEDDER.md](docs/EMBEDDER.md)** | Vector embedder architecture, MRL math, benchmarks, and atomic cutovers |
| **[docs/CONSOLIDATION.md](docs/CONSOLIDATION.md)** | Derived consolidation, prompt comparisons, and token traps |
| **[docs/NOTES-GRAIN.md](docs/NOTES-GRAIN.md)** | Notes grain taxonomy, relational schema, and 202-record legacy audit |
| **[docs/DIVERGENCES.md](docs/DIVERGENCES.md)** | Architectural divergences from the legacy NeuralSwarm orchestrator |
| **[.env.example](.env.example)** | Exhaustive configuration template with inline architectural rationale |

---

## Core Invariants: The 6 Non-Negotiable Rules

Neuralbrain was built to replace many other memory systems, which failed in four diagnosable ways (hallucinated writes, missing scope, lossy drops at 500 records, and unaddressable data). These 6 architectural rules prevent those failures from ever re-occurring:

1. **No LLM in the capture path.** Ingest records are strongly-typed columns. Rollups are pure arithmetic (counting tool invocations, summing execution times, deriving success from failure hooks). Nothing between a client request and the database row may hallucinate or summarize. (Generative models are permitted solely for producing *derived* records under Rule 6).
2. **SQLite is the source of truth. Qdrant is an index.** Every memory, observation, and trajectory must be inspectable and repairable with standard SQL. If Qdrant is wiped or corrupted, it is automatically rebuilt from SQLite.
3. **Never reject a write for capacity.** No per-session caps. No rate limiting on capture (`POST /capture/*` is permanently exempt). Backpressure is permitted; rejection is strictly forbidden. Dead letters exist solely for **malformed schema payloads** — never for volume or dependency outages.
4. **Explicit scope, never inferred — and "global" is not "unknown".** Every record is scoped to `project` (with a non-empty, folder-based `project_id`), `global` (deliberately universal across all projects), or `unknown` (unscoped, routed to the Operator Repair Queue). Never guess or backfill scope by inference.
5. **Three record types, one schema.**
   - `observations`: Machine-captured tool calls and prompts. High volume. 30-day archive, 90-day purge.
   - `trajectories`: Machine-captured task attempts with arithmetic outcome metrics. Low volume, permanent retention.
   - `notes`: Curated knowledge (facts, decisions, patterns, bugs, workflows, preferences). Authored deliberately, versioned via superseding, **never auto-deleted**.
6. **Derived records are additive, attributed, and disposable.** Summaries cite source IDs in `summary_sources`. They never mutate or replace underlying records. The entire system remains 100% functional even if every derived row is wiped.

### Database Access: One Process, One Connection (Single-Writer Invariant)

**Nothing but the Brain service opens the database file for writing.**
- The running service maintains the sole read-write connection in SQLite `WAL` mode with `foreign_keys = ON`.
- External scripts or direct SQL updates bypass the persistent `embed_queue`, corrupt vector sync, skip `failure_patterns` UPSERTs, omit tracing IDs, and risk SQLite lock contention.
- Operational repairs are performed exclusively via authenticated REST API endpoints, the Operator Dashboard, or the built-in CLI scripts when the service is idle.

---

## System Architecture

```
                              ┌────────────────────────────────────┐
                              │  Client (Agent / IDE / Framework)  │
                              │  Claude / Antigravity / Cursor...  │
                              └─────────┬────────────────┬─────────┘
                                        │                │
                         stdio / JSON-RPC│                │ HTTP REST
                                        ▼                │ (spool fallback)
                                ┌────────────────┐       │
                                │   MCP Server   │       │
                                │(Thin REST Cli) │       │
                                └───────┬────────┘       │
                                        │ HTTP REST      │
                                        ▼                ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ Neuralbrain Service (Port 3737)                                             │
│                                                                             │
│  [Middleware Pipeline]                                                      │
│    Tracing (trace_id) ──> RateLimiter (Exempt: /capture, /health) ──> Auth   │
│                                                                             │
│  [Core Endpoints]                                                           │
│    POST /capture/observations, /capture/trajectories                        │
│    POST /sessions/start, /sessions/:id/end · GET /sessions, /sessions/:id*  │
│    POST /actions · GET /actions · PATCH /actions/:id                        │
│    GET  /inject (Failure patterns, outcomes, ready actions, summary, notes) │
│    POST /search (Hybrid dense vector + in-process BM25 via RRF)             │
│    GET  /graph  (Knowledge graph across notes, sessions, actions, summaries)│
│    GET  /consolidate/candidates  &  POST /consolidate/summaries (Gate)      │
│    GET  /dashboard  (CSP-secured Operator UI)                               │
│    POST /maintenance/{reembed, rollup-sweep, cleanup, api-keys}             │
│                                                                             │
│  [Storage Engine]                            [Vector & AI Engine]           │
│    SQLite (better-sqlite3, WAL, FKs)           Embedder: nomic ONNX (local),│
│    Relational Tables (sessions, actions,        Remote Embedder, or Vendor  │
│     notes, observations, trajectories, etc.)   Matryoshka LayerNorm/Slice/L2│
│                                                Qdrant Client (Port 6333)    │
└─────────────────────────────────────────────────────────────────────────────┘
```

### Memory Layers

- **Long Term**: Curated `notes` (`fact`, `architecture`, `pattern`, `workflow`, `bug`, `preference`) and `lesson` records (`notes.type = 'lesson'`), versioned by supersede and never auto-deleted.
- **Short Term**: `sessions` lifecycle (`/sessions/start`, `/sessions/:id/end`, `/sessions/:id/replay`), `actions` (Next Actions queue via `/actions` and `brain_action_*` MCP tools), and deterministic arithmetic session summaries.
- **Observations**: High-volume `observations` (30-day archive / 90-day purge), arithmetic `trajectories`, and derived `summaries`.
- **Knowledge Graph**: Relational graph across notes, lessons, concepts, files, sessions, actions, and summaries (`GET /graph` and `GET /dashboard/api/graph`).

---

## Quickstart — `neuralbrain` CLI & `~/.neuralbrain/` (`BRAIN_HOME`)

The `neuralbrain` CLI manages installation, lifecycle, diagnostics, and agent wiring using `~/.neuralbrain/` by default (overridable via `BRAIN_HOME`):

```bash
npm install && npm run build
node dist/cli.js install   # or `neuralbrain install` once linked globally
```

`neuralbrain install` initializes `~/.neuralbrain/` (`data/brain.db`, `data/models/`, `qdrant/`, `spool/`, `.env`, `mode`) and prompts for a runtime mode:
- **`docker`**: Full Docker stack (Brain + Qdrant in Docker Compose).
- **`qdrant-docker`**: Qdrant in Docker + Brain daemon on the host (Node.js).
- **`local-only`**: Brain daemon on the host with `BRAIN_EMBED=none` (SQLite + BM25, 0 MB vector RAM).

Environment variables are loaded in priority order without clobbering existing `process.env` values: `<cwd>/.env.local` → `<cwd>/.env` → `~/.neuralbrain/.env.local` → `~/.neuralbrain/.env`.

```bash
neuralbrain install [--mode docker|qdrant-docker|local-only] [--port 3737] [--https] [--gpu] [--yes] [--no-global] [--no-agents]
neuralbrain start           # Start the configured Brain service (spawns `gpu` profile if BRAIN_EMBED=local-gpu)
neuralbrain stop            # Stop host Brain daemon and/or Docker containers
neuralbrain status          # Show BRAIN_HOME, mode, daemon PID, and /health status
neuralbrain logs            # Tail Docker Compose logs or ~/.neuralbrain/brain.log
neuralbrain doctor          # Run health and AI conformance diagnostics
neuralbrain install-agent   # Configure Antigravity, Claude Code/Desktop, Cursor, or project rules
neuralbrain mcp             # Start the stdio MCP server (11 tools)
```

> **Local HTTPS:** Passing `--https` (or setting `BRAIN_HTTPS=1`) auto-generates a local SAN certificate (`server.crt` / `server.key`) in `~/.neuralbrain/certs/` via OpenSSL and starts the Express server over `https://`.
> **Local GPU Embeddings (`--gpu`):** Passing `--gpu` sets `BRAIN_EMBED=local-gpu` (supported in `docker` and `qdrant-docker` modes), spawning the `embedder-gpu` sidecar via Docker Compose's `gpu` profile. Requires an NVIDIA GPU, host drivers, and `nvidia-container-toolkit`.

---

## Quickstart — Route A: Docker Compose (Recommended)

Docker Compose provides a zero-setup, containerized deployment including the Brain service, in-process ONNX embeddings, and the Qdrant vector database.

### 1. Configure Environment
```bash
cp .env.example .env
```
*(Optional)* Set a static API key in `.env` if you want a predetermined secret for your agents:
```dotenv
BRAIN_API_KEY=your-secret-api-key-here
```
> **Note:** If you leave `BRAIN_API_KEY` unset, the service will start in bootstrap mode, allowing you to generate and claim your first API key directly from the Operator Dashboard UI.

### 2. Start the Stack
```bash
docker compose up -d --build
```
This builds and starts two containers:
- `neuralbrain` on port `3737` (the **sole ingress point** for all agents, MCP, hooks, and the Operator Dashboard).
- `neuralbrain-qdrant` on the internal Docker network (`http://qdrant:6333`, isolated from external/agent access — only the `brain` service communicates with Qdrant).

SQLite storage is mounted to `./data` on your host.

### 3. Verify Health
```bash
docker compose ps
curl http://127.0.0.1:3737/health
```
You should receive `{"status":"ok","timestamp":...}`.

### 4. Open the Operator Dashboard
Visit [http://127.0.0.1:3737/dashboard](http://127.0.0.1:3737/dashboard) in your browser:
- **First-Time Setup**: If no key was configured in `.env`, click **`+ Claim First API Key`** to generate and claim your operator key.
- **Existing Key**: If you configured `BRAIN_API_KEY` in `.env`, paste it into the key field and click **`Apply`**.

### 5. Connect Your Coding Agents
Run the host installer to automatically wire your IDEs and agents to the Dockerized Brain:
```bash
npm install && npm run build
npm run install-agent
```

### Operational Docker Commands
```bash
# View live application logs
docker compose logs -f brain

# Run the health & AI conformance doctor
docker compose exec brain node dist/scripts/doctor.js

# Generate an additional client API key for an agent
docker compose exec brain node dist/scripts/createApiKey.js --name "claude-code"

# Migrate 202 legacy agentmemory records
docker compose exec brain node dist/scripts/migrateAgentMemory.js

# Run stdio MCP server directly inside the container
docker compose exec -i -e BRAIN_API_KEY=your-key brain node dist/mcp/server.js
```

---

## Quickstart — Route B: Standalone (Node.js on Host)

The standalone route runs directly on your host. Qdrant is **completely optional**: if you do not run Qdrant, set `BRAIN_EMBED=none` for pure SQLite + BM25 lexical search (0 MB vector RAM footprint).

### 1. Install & Build
```bash
npm install
npm run build
```

### 2. Configure Environment
```bash
cp .env.example .env
```
Choose your embedding strategy in `.env`:
- **Hybrid Dense + BM25 Search (Recommended)**: Start a local Qdrant container (`docker run -d -p 6333:6333 qdrant/qdrant:v1.18.1`) and keep `BRAIN_EMBED=local` (uses built-in ONNX `nomic-embed-text-v1.5` @ 512 dims).
- **Pure Lexical Search (Lightweight)**: Set `BRAIN_EMBED=none`. No Qdrant required.

### 3. Start the Server
```bash
# Production mode
npm start

# Development mode (auto-reload)
npm run dev
```

### 4. Open the Dashboard & Claim Key
Navigate to [http://127.0.0.1:3737/dashboard](http://127.0.0.1:3737/dashboard) and claim your API key, or paste your configured `BRAIN_API_KEY`.

### 5. Connect Coding Agents
```bash
npm run install-agent
```

---

## Switching Between Standalone and Docker (Database Portability)

Because SQLite is the absolute source of truth, moving your brain between Standalone and Docker is seamless.

- **Storage Location**:
  - Standalone: `./data/brain.db` on your local host.
  - Docker Compose: mounted via `./data:/app/data`, making `./data/brain.db` identical on both host and container!

### Migrating to an Isolated Container Volume
If you deploy to a production Docker container that uses a named volume instead of a bind mount:
```bash
# Copy host database into the Docker container
docker cp ./data/brain.db neuralbrain:/app/data/brain.db

# Copy database out of Docker container to host
docker cp neuralbrain:/app/data/brain.db ./data/brain.db
```

> **Vector Auto-Sync Guarantee**: If Qdrant is fresh, empty, or recreated, SQLite automatically drains the pending queue. To force a complete re-index of SQLite notes and trajectories into Qdrant at any time, execute:
> ```bash
> curl -X POST http://127.0.0.1:3737/maintenance/reembed \
>   -H "Authorization: Bearer <YOUR_API_KEY>"
> ```

---

## Connecting Coding Agents (`npm run install-agent`)

Neuralbrain includes an automated multi-agent installer script that detects and configures Google Antigravity, Claude Code, Claude Desktop, and Cursor.

> **Prerequisite:** Run `npm run build` once before invoking the installer.

### Safe Merging & Auto-Discovery
The installer:
- **Auto-loads `.env`**: Automatically reads `BRAIN_API_KEY` and `BRAIN_API_URL` from your `.env` file so you never need to expose credentials on the command line.
- **Safe JSON Merging**: Idempotently adds or updates `mcpServers["neuralbrain"]` without clobbering other MCP tools or configurations.
- **Hook Deduplication**: Installs lifecycle hooks (e.g. task start push injection and outcome capture) without creating duplicate entries.
- **Markdown Markers**: Context injections in `AGENTS.md` and `CLAUDE.md` are safely isolated between `<!-- neuralbrain:start -->` and `<!-- neuralbrain:end -->`.

### Installation Modes

#### 1. Global Mode (Default)
Configures all detected coding agents globally on your workstation:
```bash
npm run install-agent
```
Configures:
- **Google Antigravity**: Plugin in `~/.gemini/config/plugins/neuralbrain` + `mcp_config.json`
- **Claude**: `~/.claude/settings.json` (Claude Code) and `claude_desktop_config.json`
- **Cursor**: `~/.cursor/mcp.json`

#### 2. Project-Only Mode (`--project-dir`)
Configures memory strictly for a specific repository:
```bash
npm run install-agent -- --project-dir /path/to/my-repo
```
Configures:
- Local Claude settings: `/path/to/my-repo/.claude/settings.local.json`
- Local Cursor settings: `/path/to/my-repo/.cursor/mcp.json`
- Scoped project configuration: `/path/to/my-repo/.brainrc`
- Project context rules: Injected into `/path/to/my-repo/AGENTS.md` or `CLAUDE.md`

#### 3. Targeting Specific Agents (`--target`)
Filter installation to specific agents:
```bash
# Configure only Claude Code and Claude Desktop
npm run install-agent -- --target claude

# Configure only Google Antigravity
npm run install-agent -- --target antigravity

# Configure only Cursor
npm run install-agent -- --target cursor

# Combine specific targets
npm run install-agent -- --target claude,antigravity
```

#### 4. Dry Run Simulation
Preview all configuration modifications without writing to disk:
```bash
npm run install-agent -- --dry-run
```

---

## Remote / Cloud Deployment & AI Configuration

### Deploying Brain on a Remote Host or Cloud VM

When running the Brain service on a remote machine (e.g. LAN server, AWS EC2, GCP Cloud Run):
1. **Server Configuration**: Set `HOST=0.0.0.0` and `PORT=3737` on the remote host.
2. **Client Configuration**: On your local workstation, set `BRAIN_API_URL` to point to the remote instance:
   ```dotenv
   # .env on your local development machine:
   BRAIN_API_URL=https://brain.yourdomain.com
   BRAIN_API_KEY=your-remote-api-key
   ```
3. Run `npm run install-agent` on your local workstation. The installer automatically wires your local IDEs and agents to the remote Brain service!

### Decoupled AI Chat (`BRAIN_AI`) vs. Vector Embeddings (`BRAIN_EMBED`)

Generative chat functions and vector embeddings operate independently and can use separate endpoints:

```
┌────────────────────────────────────────────────────────────────────────┐
│                        BRAIN_ENDPOINT                                  │
│                 (Default OpenAI-compatible URL)                        │
└──────────────────┬──────────────────────────────────┬──────────────────┘
                   │                                  │
                   ▼                                  ▼
      ┌─────────────────────────┐        ┌─────────────────────────┐
      │        BRAIN_AI         │        │       BRAIN_EMBED       │
      │  (Chat / Consolidation) │        │   (Dense Vector Search) │
      ├─────────────────────────┤        ├─────────────────────────┤
      │ agent  (Host Agent MCP) │        │ local     (In-proc CPU) │
      │ none   (Disabled)       │        │ local-gpu (GPU Sidecar) │
      │ remote (Ollama/vLLM)    │        │ none      (Pure BM25)   │
      │ vendor (Cloud API Key)  │        │ remote    (Remote URL)  │
      └─────────────────────────┘        │ vendor    (Cloud Embed) │
                                         └────────────┬────────────┘
                                                      │ (Optional override)
                                                      ▼
                                         ┌─────────────────────────┐
                                         │  BRAIN_EMBED_ENDPOINT   │
                                         │ (Dedicated Embed Host)  │
                                         └─────────────────────────┘
```

#### Capability Tier Matrix

| Tier | Generative Functions (`BRAIN_AI`) | Vector Embeddings (`BRAIN_EMBED`) |
|---|---|---|
| **`none`** | Generative consolidation disabled | Pure SQLite + BM25 search (0 MB vector RAM) |
| **`agent`** | **Default**: Handled by host agent via MCP | *N/A (host agents cannot generate dense vectors)* |
| **`local`** | *N/A (Local LLMs should not run in Node process)* | **Default**: In-process CPU ONNX (`nomic` @ 512 dims) |
| **`local-gpu`** | *N/A* | Managed `embedder-gpu` sidecar via Docker Compose `gpu` profile |
| **`remote`** | Remote OpenAI-compatible chat model | External GPU-accelerated embedding endpoint |
| **`vendor`** | Cloud provider (`openai`, `anthropic`, `google`) | Cloud embedding provider with API key |

#### Managed Local GPU Embeddings (`BRAIN_EMBED=local-gpu`)
`local-gpu` runs `nomic-ai/nomic-embed-text-v1.5` in an out-of-process `embedder-gpu` container (`michaelfeil/infinity:latest`, port `7997`) without needing an external Ollama or LM Studio server:
```dotenv
BRAIN_EMBED=local-gpu
BRAIN_EMBED_MODEL=nomic
BRAIN_EMBED_DIM=512
```
**Prerequisites & Restrictions:**
- Requires an **NVIDIA GPU**, working host drivers (`nvidia-smi`), and **`nvidia-container-toolkit`** configured for Docker.
- Supported in **`docker`** and **`qdrant-docker`** modes (`neuralbrain start` spawns `--profile gpu`). In `local-only` mode, Docker is not managed by `neuralbrain` (requires a manually started server + `BRAIN_GPU_ENDPOINT`).
- First start pulls the GPU image and downloads the model into `~/.neuralbrain/data/models` (~1–2 GB VRAM, 30–120s cold start).
- Switching between `local` and `local-gpu` changes `store_meta.embed_tier` and requires `POST /maintenance/reembed`. See [docs/EMBEDDER.md](docs/EMBEDDER.md) for override variables (`BRAIN_GPU_PORT`, `BRAIN_GPU_IMAGE`, `BRAIN_GPU_ENGINE`).

#### Example: Offloading Embeddings to Remote GPU
You can generate vector embeddings on an external Ollama/vLLM server while leaving chat consolidation to your host agent:
```dotenv
BRAIN_AI=agent
BRAIN_EMBED=remote
BRAIN_EMBED_ENDPOINT=http://gpu-server.lan:11434/v1
BRAIN_EMBED_MODEL=nomic-embed-text
```

### API Key Lifecycle & Auto-Sync
- **Static Configuration**: When you set `BRAIN_API_KEY=my-key` in `.env`, the Brain service automatically registers or updates the static key on startup (`env-configured-key`).
- **Rotating Keys**: If you update `BRAIN_API_KEY` in `.env` and restart the service, the key is updated in SQLite. Simply re-run `npm run install-agent` on your development machine to refresh your agent configs.

---

## Web Operator Dashboard (`GET /dashboard`)

Accessible at `http://127.0.0.1:3737/dashboard`:
- **Security Architecture**: Zero `innerHTML` (built exclusively with safe DOM manipulation), zero `localStorage` (session state lives strictly in memory), strict Content Security Policy (`script-src 'self'`). Browser authentication uses `HttpOnly` `brain_session` cookies signed with HMAC-SHA256 and bound to a key-version derived from the active API key's stored hash (rotating or revoking a key invalidates existing dashboard sessions).
- **7 Operational Tabs**:
  1. **Overview**: High-level store metrics across observations, trajectories, notes, sessions, and actions; dead-letter and embed-queue depths; scope breakdown.
  2. **Long Term**: Browse curated notes and lessons, inspect concept/file associations, and repair `scope = 'unknown'` records with live search dropdowns.
  3. **Short Term**: Inspect active and ended `Sessions` and manage the `Next Actions` queue.
  4. **Observations**: Inspect raw `Trajectories`, `Observations`, `Summaries` with click-through source citations, and session replay timelines.
  5. **Graph**: Interactive Knowledge Graph explorer backed by `GET /dashboard/api/graph`.
  6. **Search & Inject**: Interactive dense vector + BM25 search simulator with RRF score breakdowns; live preview of the task-start `/inject` payload.
  7. **System & AI**: Visualize atomic failure patterns, inspect dead-lettered payloads, and run the automated AI conformance suite.

---

## CLI & NPM Script Reference

| Command | Description |
|---|---|
| `npm run build` | Compiles TypeScript sources (`tsc`) into `dist/` |
| `npm start` | Runs the compiled production service (`node dist/index.js`) |
| `npm run dev` | Starts development server with hot-reload (`nodemon` + `ts-node`) |
| `npm test` | Runs Jest across unit, integration, and QA test suites |
| `npm run typecheck` | Validates TypeScript types with `--noEmit` (0 errors required) |
| `npm run mcp` | Spawns the standalone stdio MCP server (`node dist/mcp/server.js`) |
| `npm run install-agent` | Installs Brain MCP and lifecycle hooks into Antigravity, Claude, and Cursor |
| `npm run doctor` | Executes operational health diagnostics and conformance tests (`--ai` / `--no-ai`) |
| `npm run smoke` | Runs end-to-end integration verification against the live server and Qdrant |
| `npm run migrate:agentmemory`| Migrates legacy 202-record agentmemory corpus into SQLite notes |
| `npm run api-key:create` | Generates a new authenticated client API key via CLI |

---

## License

MIT + Commons Clause (source-available, non-commercial resale prohibited)

---

## Contributing

Issues and PRs welcome. Please follow TypeScript + Jest conventions.