# jarvis-rag-local

**Local-first GraphRAG over a personal markdown knowledge base — with an LLM router, an MCP server and scheduled self-maintenance.**

![Python](https://img.shields.io/badge/python-3.11-blue) ![LightRAG](https://img.shields.io/badge/LightRAG-1.4-8A2BE2) ![Ollama](https://img.shields.io/badge/Ollama-local%20models-black) ![License](https://img.shields.io/badge/license-MIT-green)

> Built and operated solo, in production on a 6 GB consumer GPU since April 2026. Roughly 14 k lines of Python, one operator, zero cloud dependency by default.

---

## The problem

A consultant's business knowledge (decisions, client context, market watch, project notes) lives in ~450 markdown files. Reading them by hand doesn't scale, and pasting them into a chat model produces confident answers that are not in the notes.

**jarvis-rag-local** turns that folder into a knowledge graph you can query in natural language, with three hard constraints: every answer cites its source file, the model refuses when the corpus doesn't contain the answer, and nothing leaves the machine unless you explicitly switch a backend to a hosted API.

## Architecture

```mermaid
flowchart LR
    V[(Markdown vault<br/>~450 notes)] -->|frontmatter filter<br/>hash dedup · chunk scoring| I[scripts/ingest.py]
    I --> LR[LightRAG<br/>entity + relation extraction]
    LR --> G[(Knowledge graph<br/>14.5k entities · 12.3k relations)]
    LR --> VDB[(Vector store<br/>nano-vectordb)]
    subgraph Local models — Ollama
        Q[Qwen 2.5 3B/7B<br/>extraction + synthesis]
        E[nomic-embed-text 768d<br/>or bge-m3 1024d via sentence-transformers]
    end
    LR <--> Q
    LR <--> E
    G --> QP[Query pipeline<br/>anti-hallucination prompt v2.4<br/>citation validation]
    VDB --> QP
    QP --> CLI[scripts/query.py]
    QP --> MCP[scripts/mcp_server.py<br/>FastMCP → Claude Desktop]
    QP --> H[hermes/<br/>brain / worker router]
    H --> TG[Telegram voice bot<br/>separate repo]
    subgraph Self-maintenance
        AI[auto_ingest_weekly.py<br/>snapshot → ingest → smoke tests → rollback]
        C[agents/custodian<br/>health-check questions 3×/day]
    end
    AI --> I
    C --> QP
```

**How a query flows.** The question goes through LightRAG in `naive` mode by default (graph modes `local` / `global` / `hybrid` are available but overflow the 8k context of a 7B model on 6 GB VRAM). Retrieved chunks are handed to the synthesis model with a strict prompt: cite `[source: file.md]` for every claim, synthesise across chunks, never invent, answer in French. `scripts/query.py` then validates every citation against the real vault and strips the ones that don't exist.

**Hermes — the router.** `hermes/` splits work between a *brain* (planning, decisions, multi-step reasoning) and a *worker* (summarise, extract, classify, RAG query). Both default to local Qwen; the brain can be switched to Claude Haiku with a monthly call budget that falls back to the worker when exhausted. Short factual instructions bypass decomposition entirely and go straight to one RAG query.

**Self-maintenance.** A scheduled job snapshots the storage, ingests new or modified notes, runs three smoke queries, and rolls back automatically if they fail. A custodian agent asks the graph a rotating pool of known-answer questions three times a day and only notifies (Telegram) on anomaly.

## Stack

| Layer | Choice | Why |
|---|---|---|
| Graph + vector RAG | [LightRAG](https://github.com/HKUDS/LightRAG) 1.4 | Entity/relation graph on top of chunks, small footprint, JSON storage |
| Extraction & synthesis LLM | Qwen 2.5 3B (ingest) / 7B (query) via Ollama | Fits a 6 GB GPU; 3B is ~4× faster on extraction with marginal quality loss |
| Embeddings | nomic-embed-text (768d) or BAAI/bge-m3 (1024d) | bge-m3 is better for French but **must** run through sentence-transformers, see engineering notes |
| Hosted fallbacks | Claude Haiku 4.5, GPT, Gemini, Mistral, DeepSeek | Pluggable `--backend`; all off by default |
| Agent protocol | MCP (FastMCP, stdio) | Query the graph from Claude Desktop or any MCP client |
| Router | Custom (`hermes/`) | Task-type routing + budget guard + fallbacks |
| Ops | Python scripts + OS scheduler, Telegram notifications | No orchestrator to babysit |

## Quickstart

```bash
git clone https://github.com/LucasLeduc/jarvis-rag-local && cd jarvis-rag-local
python -m venv venv && source venv/bin/activate && pip install -r requirements.txt
ollama pull qwen2.5:3b && ollama pull qwen2.5:7b && ollama pull nomic-embed-text
cp .env.example .env            # set VAULT_PATH to a folder of .md files
python scripts/ingest.py        # first pass; add --dry-run to preview what would be ingested
python scripts/query.py "What did we decide about pricing in March?"
```

Notes are opt-in: a file is ingested only if its YAML frontmatter has `rag: true`. Folders and files listed in `EXCLUDED_FOLDERS` / `EXCLUDED_FILES` (`config.py`) are always skipped. Re-running `ingest.py` is incremental (SHA-256 per file).

Useful entry points:

| Command | What it does |
|---|---|
| `python scripts/ingest.py [--reset] [--dry-run] [--priority] [--backend ollama\|claude\|openai\|gemini\|mistral\|deepseek]` | Vault → graph. `--priority` ingests only `PRIORITY_FILES` |
| `python scripts/query.py "question" [--mode naive\|local\|global\|hybrid] [--no-adapt]` | Ask the graph, get a cited answer |
| `python scripts/mcp_server.py` | Expose `rag_query` and `rag_status` tools over MCP |
| `python -m hermes.agent "instruction" [--status] [--test]` | Route an instruction through brain/worker |
| `python scripts/auto_ingest_weekly.py [--dry-run] [--skip-smoke] [--notify-telegram]` | Snapshot → ingest → smoke tests → rollback on failure |
| `python agents/custodian/main.py [-q 3] [--notify-telegram]` | Health-check pass |
| `python scripts/eval.py --mode hybrid` | 10-question positive/adversarial eval → `eval-results/` |
| `python tools/06-snapshot_storage.py --snapshot --reason "before X"` | Manual storage snapshot (also `--list`, `--verify`, `--rotate`) |
| `python tools/08-scan_pii.py --vault /path/to/vault` | Scan the corpus for PII before ingesting |

Windows users: `scripts/setup_windows.ps1` installs Python deps, pulls the Ollama models and creates `.env`.

## Numbers (June 2026, real corpus)

| Metric | Value |
|---|---|
| Documents ingested | 462 (468 tracked, 4 failed extractions) |
| Chunks | ~1 400 (1 200 tokens, 150 overlap) |
| Entities / relations in graph | 14 572 / 12 311 |
| Storage on disk | 322 MB (JSON, snapshotted before every ingest) |
| Incremental ingest pass | 8–22 min for ~250 candidates of which 5–12 are new (rest served from hash cache), GPU RTX 6 GB |
| Query latency, local Qwen 7B, naive mode | 1–30 s depending on context size; ~11 s typical |
| Query cost | 0 € local; ≈ $0.02 per query when synthesis is switched to Claude Haiku |
| Prompt eval (April 2026, 10 questions incl. 5 adversarial) | 9/10 correct refusals or answers |
| Golden-set strict recall (May 2026, 10 questions) | 4/10 — see known limits |

The numbers above come from the state-of-the-graph reports the pipeline writes after each ingest. They are not benchmark claims; they describe one real corpus on one real machine.

## Engineering notes — things that were not in the docs

- **bge-m3 through Ollama returns NaN vectors** on French texts longer than ~300 characters (87 % NaN on real entity descriptions). The same model through `sentence-transformers` gives 0 % NaN. `embedding_st.py` exists because of that; `tools/16-test_bgem3_sanity.py` reproduces it and `tools/rebuild_vdb_st.py` re-embeds the whole store from 768d to 1024d.
- **A 7B model on 6 GB VRAM spills ~47 % to CPU.** Hybrid graph queries then time out, so `naive` is the default and the 3B model does extraction. Cheaper hardware decisions matter more than prompt tricks.
- **Graph pollution is a real failure mode.** Small extraction models create "zombie" entities (an entity named after a stray word, a person merged with a city). `tools/20-graph-cartography.py`, `21-pollution-detect.py`, `19-purge-hallucinated-entities.py` and `22-cleanup-storage.py` were written to measure and clean it; a cleanup in May 2026 removed 36 orphan document ids.
- **Refusing too much is as bad as hallucinating.** The first anti-hallucination prompt refused 6/25 legitimate questions. v2 explicitly allows multi-chunk synthesis and light inference, and separates "not in the corpus" from "phrased differently".
- **Every ingest is reversible.** Snapshot before, smoke test after, rollback on failure. Two months of daily runs, zero manual restores needed after the first week.

## Known limits

- Retrieval is sensitive to wording; the strict golden set scored 4/10 in May 2026. The reranker (`bge-reranker-v2-m3`) is declared in config but not wired in yet.
- Citation validation checks that a cited file exists, not that the sentence is actually in it.
- RAGAS scripts are present (`tools/07-run_ragas_eval_jarvis.py`) but the metrics were never fully computed on this corpus; only latency and refusal rates were.
- The synthesis prompt and the Hermes system prompt are written in French and tuned for one operator's tone. Adapting them is a copy-edit, not a refactor.
- Single-user by design: no auth, no multi-tenant storage.

## Roadmap

1. Wire the reranker and re-run the golden set.
2. Contextual chunk headers (document title + section path prepended to each chunk) to fix wording sensitivity.
3. Docker Compose (Ollama + app) and a small FastAPI layer so the MCP server and the query endpoint share one process.
4. Compute RAGAS faithfulness / context precision properly and publish the numbers, good or bad.
5. Langfuse tracing on the query path (already used on the sibling agent dashboard).

## Repository layout

```
config.py            single source of truth: paths, models, chunking, anti-hallucination prompt
local_llm.py         multi-host Ollama client with failover and short connect timeouts
embedding_st.py      sentence-transformers embedding backend (bge-m3)
scripts/             ingest, query, MCP server, eval, scheduled auto-ingest, smoke tests
hermes/              brain/worker router, usage budget, run logger
agents/custodian/    health-check agent
hooks/               Claude Code session hooks (optional memory export)
tools/               storage snapshots, PII scan, graph cartography, pollution detection, cleanup
```

## Author

Lucas Leduc — AI & automation project lead, Lille (France).
[LinkedIn](https://www.linkedin.com/in/lucas-leduc-digital/) · [GitHub](https://github.com/LucasLeduc)

Companion repositories: [jarvis-telegram-voice-bot](https://github.com/LucasLeduc/jarvis-telegram-voice-bot) (voice interface to this graph) · [jarvis-acquisition](https://github.com/LucasLeduc/jarvis-acquisition) (multi-agent outreach drafting with a learning loop) · [claude-skills](https://github.com/LucasLeduc/claude-skills).

## License

MIT — see [LICENSE](LICENSE).