# local-llm-context

**Structured SQLite entity/fact/alias memory, CSV/JSON data importer, token-budgeted per-turn context builder, chat-history normalizer, and `stdio` MCP server for small local LLMs (`4B`–`9B`).**

Zero runtime npm dependencies (powered by Node.js `22.5+` built-in `node:sqlite`). Built for **`llama.cpp` (`llama-server`)**, **Ollama**, **LM Studio**, **vLLM**, **Open WebUI**, **Continue**, **Claude Desktop**, and custom local agents.

---

## The Local LLM Data Problem

When you run a `4B`–`9B` local model (`Qwen3.5-4B`, `Llama-3.2-3B`, `Gemma-3-4B`, `Ministral-8B`) over your own structured data, two standard approaches break down:

1. **Prompt-Stuffing Overflows Context:** Pasting a 200-row JSON/CSV of accounts, servers, domains, or projects into the system prompt burns `15k`–`40k` tokens on every turn—slowing down local inference, triggering *"lost in the middle"* hallucinations, or exhausting VRAM.
2. **Vector RAG Breaks Relationships & Aliases:** Chunking tables into embeddings loses exact `parent -> child` hierarchies (*"Which domains belong to Northwind?"*), short aliases (*"NWH"* -> *"Northwind Health Group"*), and exact status counts.
3. **Background Notes Crash Strict Jinja Chat Templates:** Injecting system/event notes or replaying interrupted tool calls creates consecutive `user` roles or orphaned `tool_calls` that either overwrite assistant replies or throw `400` template errors in `llama.cpp` and Ollama.

`local-llm-context` solves all three deterministically in `< 2ms` per turn:

```mermaid
flowchart LR
    Data["CSV / JSON / Live Tool Learning"] -->|"Import & strip secrets"| DB[("SQLite ContextStore\nEntities · Aliases · Parents · Facts")]
    Prompt["User Message\ne.g. 'Any issues for NWH?'"] -->|"Word-boundary + alias\nmatcher (< 2ms)"| Builder["buildTurnContext()\nStrict char/token budget"]
    DB -->|"Matched entity +\nparent + children + facts"| Builder
    Builder -->|"~500–3,500 chars\n(only what matters)"| LLM["Local 4B–9B LLM\n(Ollama / llama.cpp)"]
```

---

## Real-World Use Cases

- **IT, Homelab & DevOps Inventory:** Import `hosts.csv`, `containers.json`, and `domains.csv` linked by parent host or account. Ask a local `4B` model *"What runs on vps-west-01?"* or *"Which services under Harborfront are degraded?"* offline.
- **Private Local CRM & Freelancer/Agency Rosters:** Store clients, contacts, aliases, projects, and dated operational facts 100% locally in SQLite—zero cloud API calls, automatic secret-key stripping (`password`, `token`, `api_key`), and instant lookup.
- **Product Catalogs, Vendors & Operations Ledgers:** Map `vendor -> SKU` or `department -> asset` tables from CSV/JSON and let the model query exact attributes or learn new facts during conversation (`entity_learn`, `note_remember`).
- **Air-Gapped Research, Legal & Clinical Notes:** Maintain structured people, organizations, matters, and chronological facts on local hardware with deterministic recall.

---

## Quick Start

### 1. Import Your CSV or JSON Data & Start the MCP Server

```bash
# Import parent accounts from CSV (with aliases and contact facts)
npx local-llm-context import \
  --db ./my-data.sqlite \
  --file ./accounts.csv \
  --kind account \
  --name account_name \
  --aliases short_code \
  --status status \
  --facts "primary_contact,renewal_date"

# Import child services/domains from JSON and auto-link them to parent accounts
npx local-llm-context import \
  --db ./my-data.sqlite \
  --file ./services.json \
  --kind service \
  --name domain \
  --parent account_name \
  --parent-kind account \
  --status status

# Inspect what exact context block a prompt receives (capped to 3,500 chars)
npx local-llm-context query \
  --db ./my-data.sqlite \
  --prompt "What services belong to NWH and who is the primary contact?"
```

Expose the SQLite store to any **MCP client** (`Claude Desktop`, `Open WebUI`, `Continue`, `Cursor`, `Zed`, or custom local agents) over `stdio`:

```json
{
  "mcpServers": {
    "local-context": {
      "command": "npx",
      "args": ["-y", "local-llm-context", "mcp", "--db", "/absolute/path/to/my-data.sqlite"]
    }
  }
}
```

Injected MCP tools:
- `store_overview` — Counts of all entities by kind, facts, and notes
- `entity_lookup` — Exact ID or filtered search returning entity, parent, children, attributes, and facts
- `entity_learn` — Upsert entities, aliases, parent links, status, attributes, or durable facts
- `note_remember` / `note_recall` — Key-value operational notes
- `context_for_prompt` — Build a budgeted context block for any prompt string

---

### 2. Use in TypeScript / Node.js with `llama.cpp` or Ollama

```ts
import {
  ContextStore,
  appendBackgroundNote,
  budgetChatHistory,
  buildTurnContext,
  executeContextTool,
  getContextToolSpecs,
  importJson,
   type ChatTurnMessage,
} from 'local-llm-context';

const store = new ContextStore('./my-data.sqlite');

// 1. Load structured JSON or CSV once (secrets like api_key/password are stripped automatically)
importJson(
  store,
  [
    { name: 'Northwind Health', code: 'NWH', status: 'active', contact: 'dana@example.org' },
  ],
  { kind: 'account', nameField: 'name', aliasFields: ['code'], factFields: ['contact'] },
);

// 2. Per turn: inject ONLY the matched entities (+ parents/children/facts) within a strict budget
const userPrompt = 'Who is our contact at NWH?';
const turnContext = buildTurnContext(userPrompt, store, { budgetChars: 3_500 });

let history: ChatTurnMessage[] = [
  { role: 'system', content: ['You are a concise local assistant.', turnContext].filter(Boolean).join('\n\n') },
  { role: 'user', content: userPrompt },
];

// 3. Safely append background/event notes without overwriting assistant replies or breaking Jinja templates
history = appendBackgroundNote(history, 'Backup job completed at 03:00 UTC.', 'Background Event');

// 4. Normalize orphaned tool calls, strip old <think> traces, and compact stale tool payloads before calling Ollama / llama-server
const safeMessages = budgetChatHistory(history, {
  maxHistoryChars: 48_000,
  staleToolMaxChars: 2_000,
  freshToolMaxChars: 16_000,
});

const response = await fetch('http://127.0.0.1:8080/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'qwen-local',
    messages: safeMessages,
    tools: getContextToolSpecs().map((t) => ({
      type: 'function',
      function: { name: t.name, description: t.description, parameters: t.parameters },
    })),
  }),
});
```

---

## Companion Project

Need your small local LLM (`4B`–`9B`) to process **large 100+ row tool outputs across multiple turns** without hitting `max_tokens` output cutoffs? Pair `local-llm-context` with [**`local-llm-relay`**](https://github.com/cynthiaschomp/local-llm-relay).

---

## Development & Tests

Requires **Node.js `>= 22.5.0`**.

```bash
npm install
npm test
```

## License

MIT © Cynthia Schomp