# ctxshard — Deterministic SQLite Memory & Pre-Turn Context Compiler for Local LLMs (4B–14B)

> **`ctxshard`** is a zero-dependency, physically sharded multi-tenant memory engine, state graph, and `stdio` MCP server built on Node.js native `node:sqlite` (`FTS5` + `WAL`). Instead of running vector databases or hoping a `4B`–`9B` small language model remembers to call `search_memory`, `ctxshard` deterministically compiles matching entities, aliases, relational assertions, directives, and live resource state from isolated `<tenant>.sqlite` files into a `<800`-token prompt frame in `~1ms`.

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
[![Node.js >= 22.5](https://img.shields.io/badge/node-%3E%3D22.5.0-brightgreen.svg)](https://nodejs.org/)
[![Zero Dependencies](https://img.shields.io/badge/dependencies-0-success.svg)](./package.json)
[![MCP Ready](https://img.shields.io/badge/MCP-stdio-purple.svg)](https://modelcontextprotocol.io/)

Works out of the box with **`llama.cpp` (`llama-server`)**, **Ollama**, **LM Studio**, **vLLM**, **Claude Desktop**, **Cursor**, **OpenCode**, **Goose**, and custom local agent loops.

---

## Why Small Local Models (`4B`–`9B`) Break on Standard AI Memory

Most agent memory frameworks (`Letta`, `Mem0`, `Zep / Graphiti`, vector RAG) assume frontier reasoning budgets or external embedding infrastructure. When paired with local SLMs (`Qwen3.5-4B`, `Qwen2.5-7B`, `Llama-3.2-3B/8B`, `Gemma-3-4B`, `Phi-4`, `Ministral-8B`), four failure modes appear immediately:

1. **Skipped Memory Tool Calls:** `4B`–`9B` models frequently answer directly without calling `recall()` or `search_nodes()` first, hallucinating missing context.
2. **Vector RAG Pollution in `8k`–`16k` Windows:** Cosine-similarity chunks inject noisy, low-precision text into narrow local KV caches while burning extra RAM/VRAM on embedding models (`sqlite-vec`, `Chroma`, `sentence-transformers`).
3. **Cross-Tenant Context Bleed:** Storing personal, employer, and client projects inside one shared vector index or JSONL file risks leaking facts across workspaces via fuzzy similarity.
4. **Restart Amnesia & `node-gyp` Friction:** Live agent state maps vanish on process restart, and C++ SQLite addons (`better-sqlite3`) frequently fail to compile across Node upgrades.

`ctxshard` fixes all four in deterministic CPU code using Node's built-in `node:sqlite`—zero runtime npm dependencies, zero embeddings, and hard file-per-tenant isolation.

---

## How `ctxshard` Compares to Existing Agent Memory Tools

| Capability | `ctxshard` | `Letta (MemGPT)` | `Mem0` | `Zep / Graphiti` | `@modelcontextprotocol/server-memory` |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **Works if `4B` model skips tool call** | **Yes (Pre-turn CPU compiler)** | No (Requires self-paging tool loops) | Partial | Partial | No (Passive pull-only) |
| **Vector DB / Embedding VRAM needed** | **No (`0 MB` — SQLite `FTS5` + Graph)** | Yes | Yes | Yes + Graph DB | No (Flat JSONL file) |
| **Hard multi-tenant isolation** | **1 `<tenant>.sqlite` file per tenant** | Shared DB | Logical namespace | Logical namespace | Single shared JSONL |
| **Live resource state + metric merge** | **Built-in (`resources` table)** | No | No | Temporal edges | No |
| **Runtime npm / C++ dependencies** | **Zero (`node:sqlite` built-in)** | Python server + DB | Vector store + SDK | Neo4j / FalkorDB | Node (JSONL lock risk) |
| **`4B`–`9B` local SLM reliability** | **High** | Low (`<14B` paging loops) | Medium | Medium | Medium |

---

## Architecture

```text
User Prompt: "Why is ACME slow right now?"
      │
      ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Pre-Turn Context Compiler (compileContext — ~1.0ms CPU, 0 LLM tokens)   │
│  1. Boundary-safe entity + alias linker ("ACME" -> client:acme-core)    │
│  2. SQLite FTS5 keyword fallback across entities_fts & assertions_fts   │
│  3. 1-hop graph walk (parent, children, linked live resource telemetry) │
│  4. Priority packer with hard token ceiling (default <= 800 tokens)     │
└───────────────┬─────────────────────────────────────────┬───────────────┘
                │                                         │
                ▼                                         ▼
┌───────────────────────────────┐         ┌───────────────────────────────┐
│ principal.sqlite (Global)     │         │ shards/acme.sqlite (Isolated) │
│ • Principal identity & role   │         │ • entities + entities_fts     │
│ • Tenant shard registry       │         │ • assertions + assertions_fts │
│ • Append-only audit_log       │         │ • resources (live state+meta) │
└───────────────────────────────┘         │ • directives (standing KV)    │
                                          └───────────────────────────────┘
                │
                ▼
Injected System Frame (~245 tokens) ──► Local 4B–9B Model (llama.cpp / Ollama)
```

```mermaid
flowchart LR
    Prompt["User Prompt"] --> Compiler["Deterministic Context Compiler\n(~1ms CPU · Hard Token Budget)"]
    Principal[("principal.sqlite\nIdentity + Audit Log")] --> Compiler
    ShardA[("shards/acme.sqlite\nEntities · FTS5 · State")] --> Compiler
    ShardB[("shards/personal.sqlite\nPhysically Sealed")] -. "Air-gapped when\nacme is active" .- Compiler
    Compiler --> Frame["Compact Context Frame\n(<= 800 tokens)"]
    Frame --> SLM["Local 4B–14B LLM\n(llama.cpp / Ollama / vLLM)"]
```

---

## Quick Start

### 1. Drop-In `stdio` MCP Server (Claude Desktop, Cursor, OpenCode, Goose)

Add `ctxshard` to your `.mcp.json` or `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "ctxshard": {
      "command": "npx",
      "args": ["-y", "ctxshard", "mcp"]
    }
  }
}
```

Optional flags: `--dir /custom/path/to/data` and `--tenant my-workspace`.

### 2. Pre-Turn Context Compilation in TypeScript (`llama.cpp` / Ollama / OpenAI-Compatible)

Install into any Node.js `>=22.5.0` project:

```bash
npm install ctxshard
```

Compile deterministic tenant memory before calling your local model—so even a `4B` model that never calls a memory tool still answers with exact facts and live state:

```ts
import { CtxShardEngine } from 'ctxshard';

const memory = new CtxShardEngine({
  defaultTenant: 'acme',
  principal: {
    name: 'Jordan Lee',
    role: 'Staff Infrastructure Engineer',
    directives: ['Never cross-pollinate client data', 'Cite live resource status first'],
  },
});

// 1. Teach durable entities + plain-sentence assertions (scoped to shards/acme.sqlite)
memory.upsertEntity({
  id: 'client:acme',
  kind: 'client',
  name: 'Acme Logistics',
  aliases: ['ACME'],
  about: 'Primary freight routing platform',
  resourceId: 'site:api.acme.internal',
  fact: 'Runs PostgreSQL 17 on us-west-2; maintenance window is Tuesdays 02:00 UTC.',
});

// 2. Persist live resource state (telemetry polls merge metrics without wiping learned tags)
memory.upsertResources([
  {
    id: 'site:api.acme.internal',
    kind: 'site',
    name: 'api.acme.internal',
    status: 'warn',
    metrics: { tier: 'enterprise', latencyMs: 940, sslDays: 62 },
  },
]);

// 3. Compile a token-bounded frame in ~1ms before calling llama-server / Ollama
const userPrompt = 'Why is ACME degraded right now, and when can we fail over Postgres?';
const frame = memory.compileContext(userPrompt, { maxTokens: 600 });

const res = await fetch('http://127.0.0.1:8080/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'qwen-local',
    messages: [
      { role: 'system', content: `Use this verified workspace context:\n\n${frame.text}` },
      { role: 'user', content: userPrompt },
    ],
  }),
});
```

### 3. Human CLI Inspector (Zero Black-Box Embeddings)

Every shard is plain relational SQLite. Inspect, teach, export, or physically purge any tenant from your terminal:

```bash
# Seed an entity + fact from CLI
npx ctxshard entity add client "Acme Logistics" --tenant acme --about "Freight platform" --fact "Bills quarterly"

# Preview the exact compiled prompt frame and token count
npx ctxshard compile "When does Acme Logistics bill?" --tenant acme --max-tokens 600

# Atomic single-file backup or complete physical deletion (GDPR / client offboarding)
npx ctxshard tenant export acme ./backups/acme-2026.sqlite
npx ctxshard tenant purge acme
```

---

## Benchmarks (`npm run bench`)

Measured locally on Node.js (`node:sqlite` WAL + `PRAGMA data_version` write-invalidated cache) across **1,000 seeded entities + 1,000 assertions + 1,000 live resources**:

| Metric | Result |
| :--- | :--- |
| **Seed 1,000 entities + FTS5 + resources** | `~370 ms` |
| **`compileContext()` median latency (`p50`)** | **`1.03 ms`** |
| **`compileContext()` tail latency (`p99`)** | **`2.75 ms`** |
| **Compiled prompt size (2 matched entities + principal + tail)** | **`~245 tokens`** (`<= 800` hard cap) |
| **Extra embedding / vector DB VRAM required** | **`0 MB`** |
| **Runtime npm dependencies** | **`0`** |

Run `npm run bench` on your own hardware to reproduce.

---

## MCP Tools & Resources Exposed

When running `ctxshard mcp`, the following small-model-safe tools and resources are exposed:

- **`compile_context`** — Deterministically compile a token-budgeted context frame for a prompt.
- **`tenant_switch` / `tenant_list`** — Switch active `<tenant>.sqlite` shard or inspect shard counts.
- **`entity_upsert` / `entity_lookup`** — Upsert typed graph nodes (`person`, `company`, `brand`, `client`, `project`, `service`, `place`, `other`) or search via SQLite `FTS5`.
- **`assertion_add`** — Attach deduplicated plain-sentence facts with provenance (`human | agent | ingest`).
- **`resource_upsert` / `resource_list`** — Persist tracked infrastructure/business resources (`client`, `site`, `service`, `machine`, `agent`, `lead`, `ticket`) across restarts while merging live telemetry metrics cleanly.
- **`directive_set` / `directive_recall`** — Store and search per-tenant standing rules and preferences.
- **Resource `ctxshard://context/active`** — Live text snapshot of the active tenant shard.

---

## Pairing with `local-llm-relay`

Building full autonomous workflows on `4B`–`9B` local models? Pair **`ctxshard`** (long-term sharded memory & pre-turn context compilation) with [**`local-llm-relay`**](https://github.com/cynthiaschomp/local-llm-relay) (out-of-context SQLite dataset handles `ds-*` for 200+ row tool outputs, persistent multi-turn checklists, and `max_tokens` batch-shrinking recovery).

---

## FAQ

### How do I give an Ollama or `llama.cpp` 4B model long-term memory without a vector database?
Install `ctxshard`, record entities/assertions via `entity_upsert` (or the `stdio` MCP server), and call `engine.compileContext(userMessage, { maxTokens: 600 })` before sending your chat completion request. Relevant entities, aliases, facts, and live resource statuses are matched in `<2ms` via word-boundary linking and SQLite `FTS5`—requiring zero vector embeddings.

### How does multi-tenant isolation work for agencies, consultants, or work-vs-personal setups?
Each tenant ID maps to a separate SQLite database file on disk (`~/.local/share/ctxshard/shards/<tenant_id>.sqlite`). Queries against tenant `client-a` never open or scan `client-b.sqlite` or `personal.sqlite`. Syncing a single workspace to a work laptop or permanently wiping an offboarded client is a single file copy (`tenant export`) or file delete (`tenant purge`).

### Why use SQLite `FTS5` + an entity graph instead of vector embeddings for SLMs?
Exact identifiers—hostnames, error codes, account slugs, ports, SKU names, and policy constraints—degrade under dense vector compression and return fuzzy near-misses that confuse small `4B`–`9B` models. Relational entity/alias matching + `FTS5` prefix search is exact, deterministic, auditable in plain SQL, and uses `0 MB` of GPU memory.

---

## License

MIT © Cynthia Schomp