Skip to main content
Glama

local-llm-context

Structured SQLite entity/fact/alias memory, CSV/JSON data importer, token-budgeted per-turn context builder, chat-history normalizer, and stdio MCP server for small local LLMs (4B–9B).

Zero runtime npm dependencies (powered by Node.js 22.5+ built-in node:sqlite). Built for llama.cpp (llama-server), Ollama, LM Studio, vLLM, Open WebUI, Continue, Claude Desktop, and custom local agents.


The Local LLM Data Problem

When you run a 4B–9B local model (Qwen3.5-4B, Llama-3.2-3B, Gemma-3-4B, Ministral-8B) over your own structured data, two standard approaches break down:

  1. Prompt-Stuffing Overflows Context: Pasting a 200-row JSON/CSV of accounts, servers, domains, or projects into the system prompt burns 15k–40k tokens on every turn—slowing down local inference, triggering "lost in the middle" hallucinations, or exhausting VRAM.

  2. Vector RAG Breaks Relationships & Aliases: Chunking tables into embeddings loses exact parent -> child hierarchies ("Which domains belong to Northwind?"), short aliases ("NWH" -> "Northwind Health Group"), and exact status counts.

  3. Background Notes Crash Strict Jinja Chat Templates: Injecting system/event notes or replaying interrupted tool calls creates consecutive user roles or orphaned tool_calls that either overwrite assistant replies or throw 400 template errors in llama.cpp and Ollama.

local-llm-context solves all three deterministically in < 2ms per turn:

flowchart LR
    Data["CSV / JSON / Live Tool Learning"] -->|"Import & strip secrets"| DB[("SQLite ContextStore\nEntities · Aliases · Parents · Facts")]
    Prompt["User Message\ne.g. 'Any issues for NWH?'"] -->|"Word-boundary + alias\nmatcher (< 2ms)"| Builder["buildTurnContext()\nStrict char/token budget"]
    DB -->|"Matched entity +\nparent + children + facts"| Builder
    Builder -->|"~500–3,500 chars\n(only what matters)"| LLM["Local 4B–9B LLM\n(Ollama / llama.cpp)"]

Related MCP server: Kremis

Real-World Use Cases

  • IT, Homelab & DevOps Inventory: Import hosts.csv, containers.json, and domains.csv linked by parent host or account. Ask a local 4B model "What runs on vps-west-01?" or "Which services under Harborfront are degraded?" offline.

  • Private Local CRM & Freelancer/Agency Rosters: Store clients, contacts, aliases, projects, and dated operational facts 100% locally in SQLite—zero cloud API calls, automatic secret-key stripping (password, token, api_key), and instant lookup.

  • Product Catalogs, Vendors & Operations Ledgers: Map vendor -> SKU or department -> asset tables from CSV/JSON and let the model query exact attributes or learn new facts during conversation (entity_learn, note_remember).

  • Air-Gapped Research, Legal & Clinical Notes: Maintain structured people, organizations, matters, and chronological facts on local hardware with deterministic recall.


Quick Start

1. Import Your CSV or JSON Data & Start the MCP Server

# Import parent accounts from CSV (with aliases and contact facts)
npx local-llm-context import \
  --db ./my-data.sqlite \
  --file ./accounts.csv \
  --kind account \
  --name account_name \
  --aliases short_code \
  --status status \
  --facts "primary_contact,renewal_date"

# Import child services/domains from JSON and auto-link them to parent accounts
npx local-llm-context import \
  --db ./my-data.sqlite \
  --file ./services.json \
  --kind service \
  --name domain \
  --parent account_name \
  --parent-kind account \
  --status status

# Inspect what exact context block a prompt receives (capped to 3,500 chars)
npx local-llm-context query \
  --db ./my-data.sqlite \
  --prompt "What services belong to NWH and who is the primary contact?"

Expose the SQLite store to any MCP client (Claude Desktop, Open WebUI, Continue, Cursor, Zed, or custom local agents) over stdio:

{
  "mcpServers": {
    "local-context": {
      "command": "npx",
      "args": ["-y", "local-llm-context", "mcp", "--db", "/absolute/path/to/my-data.sqlite"]
    }
  }
}

Injected MCP tools:

  • store_overview — Counts of all entities by kind, facts, and notes

  • entity_lookup — Exact ID or filtered search returning entity, parent, children, attributes, and facts

  • entity_learn — Upsert entities, aliases, parent links, status, attributes, or durable facts

  • note_remember / note_recall — Key-value operational notes

  • context_for_prompt — Build a budgeted context block for any prompt string


2. Use in TypeScript / Node.js with llama.cpp or Ollama

import {
  ContextStore,
  appendBackgroundNote,
  budgetChatHistory,
  buildTurnContext,
  executeContextTool,
  getContextToolSpecs,
  importJson,
   type ChatTurnMessage,
} from 'local-llm-context';

const store = new ContextStore('./my-data.sqlite');

// 1. Load structured JSON or CSV once (secrets like api_key/password are stripped automatically)
importJson(
  store,
  [
    { name: 'Northwind Health', code: 'NWH', status: 'active', contact: 'dana@example.org' },
  ],
  { kind: 'account', nameField: 'name', aliasFields: ['code'], factFields: ['contact'] },
);

// 2. Per turn: inject ONLY the matched entities (+ parents/children/facts) within a strict budget
const userPrompt = 'Who is our contact at NWH?';
const turnContext = buildTurnContext(userPrompt, store, { budgetChars: 3_500 });

let history: ChatTurnMessage[] = [
  { role: 'system', content: ['You are a concise local assistant.', turnContext].filter(Boolean).join('\n\n') },
  { role: 'user', content: userPrompt },
];

// 3. Safely append background/event notes without overwriting assistant replies or breaking Jinja templates
history = appendBackgroundNote(history, 'Backup job completed at 03:00 UTC.', 'Background Event');

// 4. Normalize orphaned tool calls, strip old <think> traces, and compact stale tool payloads before calling Ollama / llama-server
const safeMessages = budgetChatHistory(history, {
  maxHistoryChars: 48_000,
  staleToolMaxChars: 2_000,
  freshToolMaxChars: 16_000,
});

const response = await fetch('http://127.0.0.1:8080/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'qwen-local',
    messages: safeMessages,
    tools: getContextToolSpecs().map((t) => ({
      type: 'function',
      function: { name: t.name, description: t.description, parameters: t.parameters },
    })),
  }),
});

Companion Project

Need your small local LLM (4B–9B) to process large 100+ row tool outputs across multiple turns without hitting max_tokens output cutoffs? Pair local-llm-context with local-llm-relay.


Development & Tests

Requires Node.js >= 22.5.0.

npm install
npm test

License

MIT © Cynthia Schomp

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      Not graded
      maintenance
      A Knowledge Graph MCP server optimized for LLM context efficiency through compact JSON and SQLite persistence. It enables full graph management including node/edge CRUD operations, full-text search, and subgraph traversal.
      -
    • A
      license
      A
      quality
      A
      maintenance
      MCP server exposing a deterministic, local knowledge graph over stdio. Zero LLM calls in the bridge; answers are classified as Fact, Inference, or Unknown and persisted in redb (ACID, BLAKE3-hashed).
      10
      16
      Apache 2.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables coding agents to keep a local knowledge graph across sessions and checkpoints, restoring project context after tabs or model changes, and to ground every answer with an explicit verdict from the graph. Runs entirely on the user's machine over MCP/stdio or local JSON-RPC with no third-party runtime dependencies.
      13
      Apache 2.0