# Bigos — a local RAG that eats whatever you throw in the pot

> **B**ilingual **I**ndex **G**enerator for **O**ffline **S**earch. Also a
> Polish stew where everything ends up in one pot, which is exactly how this
> works: PDFs, ebooks, Yocto recipes, C headers, Polish, English — one
> collection, no language filter. Like the dish, it is better the second day,
> once the cache is warm.

A fully local retrieval index over documents **and source trees**. Claude acts
as the orchestrator and the data never leaves the machine: embeddings go to
Ollama running natively on the host, vectors live in Qdrant inside a rootless
container, and Claude talks to it through an MCP server over stdio.

**The server does not generate anything.** It tokenises, embeds and retrieves;
the reasoning happens in the orchestrating model. See
[Why there is no generation here](#why-there-is-no-generation-here).

## Architecture

```
Claude Code ──stdio/JSON-RPC──▶ bigos MCP server (venv, host)
                                    │
                    ┌───────────────┴───────────────┐
                    ▼                               ▼
        Ollama (native, host)              Qdrant (rootless Podman)
        └─ bge-m3 → GPU, 4 GB VRAM         127.0.0.1:6333
           keep_alive: -1                  collection: bigos
```

One model, one job. bge-m3 is loaded into the 4 GB of VRAM once and never
evicted (`keep_alive: -1`); nothing else on this host asks the GPU for
anything. There are no generative models in the data path at all.

## Requirements

| Component | Version / notes |
|---|---|
| Arch Linux, rootless Podman | `podman` 6.x, `subuid`/`subgid` configured |
| Ollama | installed **natively** (not in a container), with GPU access |
| Disk | ~1.2 GB for bge-m3, plus the index (50 MB per ~3600 chunks) |
| Python | 3.12+ (3.14 on this host — every dependency ships `cp314` wheels) |
| Optional: OCR | `poppler` + `tesseract` + `tesseract-data-pol` `tesseract-data-eng` |
| Optional: MOBI | the `mobi` package from `requirements.txt` (or `calibre` as a fallback) |

## Installation

```bash
./setup.sh                      # full setup, asks before large downloads
./setup.sh --check              # diagnostics only, changes nothing
./setup.sh --yes --skip-models  # without the ~11 GB of models
```

Useful flags: `--with-ocr` (tesseract language data via `sudo pacman`),
`--with-ollama-dropin` (systemd drop-in with `OLLAMA_KEEP_ALIVE=-1`),
`--with-deepseek` (also pull `deepseek-coder-v2:16b`), `--enable-linger`
(start Qdrant without a logged-in session), `--with-rtk-filters` (`rtk trust`).

The script is idempotent — run it as many times as you like.

## Usage

### Indexing a corpus

```bash
.venv/bin/python -m bigos.ingest ~/documents --recursive
.venv/bin/python -m bigos.ingest ~/ebooks/scan.pdf --ocr always
.venv/bin/python -m bigos.ingest ~/notes --include '*.md' --dry-run
```

Files whose content has not changed since the last run are skipped by content
hash, so re-scanning a large folder costs a read per file rather than a full
re-embedding. `--reindex` forces the work anyway; `--prune` also forgets
documents under that path whose source file has been deleted.

### Removing things from the index

The chunk text lives in the Qdrant payload, so the index is a readable copy of
whatever you fed it. Removing a document from the index is what actually takes
its content off this machine's search surface.

```bash
.venv/bin/python -m bigos.ingest ~/bigos-inbox/secret.pdf --forget   # one file
.venv/bin/python -m bigos.ingest ~/projects/client-x --forget        # a whole tree
curl -X DELETE http://127.0.0.1:6333/collections/bigos               # the lot
```

`--forget` works whether or not the source file still exists. Where the data
sits on disk:

| Path | What it holds |
|---|---|
| `~/.local/share/bigos/qdrant/storage` | vectors **and the chunk text** |
| `~/.cache/bigos/ocr` | text recognised from scanned pages, per content hash |
| `models/tokenizer/bge-m3-tokenizer.json` | the tokenizer; `setup.sh` re-downloads it |

Nothing is written anywhere else: no `~/.cache/huggingface`, no temp files that
outlive a run, nothing outside the paths above.

### The drop folder

Install the watcher once:

```bash
./setup.sh --with-inbox-watcher                 # default folder: ~/bigos-inbox
./setup.sh --with-inbox-watcher --inbox ~/books # or point it anywhere
```

From then on, dropping a file into that folder indexes it a few seconds later,
and deleting a file from it removes it from the knowledge base. Two systemd
user units do the work: `bigos-ingest.path` watches the folder with inotify
`IN_CLOSE_WRITE` (so a half-copied file cannot trigger a run), and
`bigos-ingest.service` waits until the folder stops changing before ingesting.

```bash
journalctl --user -u bigos-ingest -f      # watch it work
systemctl --user start bigos-ingest       # run it by hand
systemctl --user disable --now bigos-ingest.path
```

The watch is not recursive: creating a file deep inside a subdirectory does not
trigger a run on its own, though the ingestion itself walks subdirectories, so
the next trigger picks it up.

Supported documents: **PDF** (with OCR for scans), **Markdown**, **TXT**,
**EPUB**, **MOBI/AZW3/AZW/PRC**, **HTML**.

### Source trees

```bash
.venv/bin/python -m bigos.ingest ~/yocto/meta-mylayer --recursive
.venv/bin/python -m bigos.ingest ~/src/firmware --exclude vendor --exclude 3rdparty
```

Indexed as code: **C/C++** (`.c .h .cpp .cc .cxx .hpp .hh`), **Python**,
**Bash/Zsh**, **BitBake/Yocto** (`.bb .bbappend .bbclass .inc`), **Make**,
**CMake**, **Meson**, **Rust**, **Go**, **Verilog/SystemVerilog/VHDL**,
**device tree** (`.dts .dtsi .dtso`), **assembly** (`.S .s`), **linker scripts**
(`.lds`), **reST docs** (`.rst`), **YAML/TOML/INI**, plus build files with no
extension (`Makefile`, `CMakeLists.txt`, `Dockerfile`, `Kconfig`, `Kbuild`) and
the ones only a glob catches: `Kconfig.debug`, `Makefile.lib`, `Kbuild.include`,
`imx8mm_evk_defconfig`.

### Kernel and bootloader trees

```bash
# A subsystem, not the whole tree
.venv/bin/python -m bigos.ingest ~/linux/drivers/spi --recursive
.venv/bin/python -m bigos.ingest ~/u-boot/board/freescale --recursive
.venv/bin/python -m bigos.ingest ~/linux/Documentation/spi --recursive
```

Verified against real sources: a Polish question about clock dividers returns
`spi-imx.c:1348-1398`, an English one about arm64 exception vectors returns
`entry.S:1-46`, and `.rst` documentation, `Kconfig` and `_defconfig` files are
all searchable alongside the C.

**Scope the path.** A full Linux checkout is ~80k indexable files and millions
of chunks - days of embedding on a laptop GPU. `--max-files` (default 5000)
refuses to start above that and says so, rather than quietly running until
Tuesday. Run `--dry-run` first: it reports the chunk count and an estimated
embedding time before you commit.

Code is tokenised and chunked exactly like prose - there is no parser and no
AST. What you get back is the file and the **line range**, which is what makes
a hit actionable:

```
0.5878 [code/bitbake] nativesdk-mingw-w64-runtime_9.0.0.bb:1-28
```

Natural-language detection is skipped for code, because running a PL/EN
detector over C or BitBake produces noise rather than metadata.

**Pruning matters more than anything else here.** The walk skips
`.git`, `__pycache__`, `node_modules`, `.venv`, `dist`, `build`, `target` and
the Yocto build output (`tmp`, `tmp-glibc`, `sstate-cache`, `downloads`,
`deploy`, `buildhistory`) by pruning the directory, not by filtering afterwards.
Add your own with `--exclude`. Files above 1 MB and anything with a NUL byte in
its first 8 KB are skipped as generated or binary.

A word of warning learned the hard way: do not guess directory names. `workdir`
looks like build output and in at least one real project holds the layer
checkouts - excluding it silently dropped 99% of the corpus.

Re-indexing the same file overwrites its chunks instead of duplicating them:
point IDs are deterministic, and a document's previous chunks are deleted
before the new ones are written.

### MCP tools

Restart Claude Code in this directory and the `bigos` server from `.mcp.json`
connects automatically (check with `/mcp`).

| Tool | Purpose |
|---|---|
| `search_knowledge_base(query, top_k, kind, max_chars_per_result)` | cross-lingual retrieval over documents and code |
| `ingest_document(path, recursive, ocr, reindex)` | index a file or a directory |

`kind` narrows a search to `"code"` or `"document"`. It is the one filter that
exists: prose and source code answer different questions, while natural
language is deliberately never filtered on.

`search_knowledge_base` returns raw fragments (trimmed to 800 characters each)
with their provenance: file path, page number for PDFs, **line range for code**,
heading trail for structured documents.

## Why there is no generation here

An earlier version had `analyze_code` and `generate_code` running a local model
on the CPU. Measured on this host (i5-12500H), asked to write an FSM in
SystemVerilog with the project's own Intel FPGA guide retrieved into the prompt:

| Task | qwen2.5-coder:7b Q4 | deepseek-coder-v2:16b (MoE) |
|---|---|---|
| 5-line SVA assertion | 83 s, correct | - |
| ~150-line FSM module | 226 s, does not compile (5 states in an `enum logic [1:0]`, latches, a combinational loop) | 122 s, compiles and follows the retrieved coding style |

The stronger model fixed every *conformance* defect and none of the *design*
ones: its 8-floor elevator had no current-floor register, and its 8-bit counter
never used its direction input, so `COUNT_DOWN` was unreachable.

That is the whole argument. Retrieval is where a local machine wins - it is
fast, private and exact. Generation over a whole module is where it loses, and
routing it through a weaker model added nothing that the orchestrating model
does not do better. So the server retrieves, and only retrieves.

The lesson: a 4-bit quantised 7B model holds up for short, local answers and
falls apart over a whole module, even when retrieval hands it the exact coding
guideline it should follow. For modules, either let the orchestrating model
write the code from the retrieved sources, or switch to a stronger local model
(`deepseek-coder-v2:16b` is a MoE with ~2.4B active parameters, so it stays
usable on a CPU).

## How cross-lingual search works

The whole corpus, whatever its language, goes into **one collection and one
vector space** of the multilingual bge-m3 model. The detected language is
stored in every chunk's metadata, but **no query ever filters on it**. That is
what makes a Polish question return English fragments and the other way round.
It is the entire mechanism — there is no query translation and no per-language
index.

## Chunking

A 512-token window with a 64-token overlap, measured with the **bge-m3
tokenizer** (`models/tokenizer/bge-m3-tokenizer.json`, downloaded once by
`setup.sh`, fully offline afterwards). Token boundaries are mapped back to
character offsets, so what lands in Qdrant is the original text, not a
detokenised reconstruction.

Polish text breaks into noticeably more tokens than English of the same
character length — which is why we count tokens rather than characters.

Chunk metadata: `kind` (`document` or `code`), `code_language` (`c`, `python`,
`bitbake`, …), `language` (document level, prose only), `chunk_language` (the
fragment's own language, which differs in bilingual books), `source_path`,
`source_name`, `page`, `page_end`, `line_start`, `line_end`, `heading_path`,
`chunk_index`, `char_start`, `char_end`, `token_count`, `content_hash`,
`ingested_at`.

## OCR

`ocrmypdf` is not packaged for Arch, and the pipeline needs text rather than a
searchable PDF anyway. So pages without a text layer are rendered with
`pdftoppm` (300 dpi) and recognised with `tesseract` (`pol+eng`), four pages at
a time. Results are cached under `~/.cache/bigos/ocr`, keyed by the
file's content hash, so re-ingesting the same scan is instant.

```bash
sudo pacman -S poppler tesseract tesseract-data-pol tesseract-data-eng
```

Policies: `auto` (default — only pages without text), `never`, `always`.

## Keeping the GPU in shape

```bash
nvidia-smi                      # bge-m3 should occupy ~1.3 GB of VRAM
ollama ps                       # which model sits on GPU vs CPU, and for how long
journalctl -u ollama -n 50      # which model was loaded
```

Every embedding request sends `keep_alive: -1`, so the model stays in VRAM. The
systemd drop-in (`--with-ollama-dropin`) is optional — it only matters right
after a restart of `ollama.service`, before the first request loads the model.

bge-m3 is the only model this project needs; `./scripts/pull_models.sh` pulls
just that (~1.2 GB).

## Diagnostics

```bash
.venv/bin/python scripts/healthcheck.py           # status table
.venv/bin/python scripts/healthcheck.py --warmup  # + load bge-m3 into VRAM
.venv/bin/python scripts/smoke_test.py            # cross-lingual test
.venv/bin/python -m pytest tests/ -q              # offline tests
systemctl --user status qdrant
journalctl --user -u qdrant -n 50
```

An unreachable Qdrant or Ollama never ends in a traceback: both the MCP tools
and the CLI return a single sentence with the command that fixes it.

## RTK integration

The `PreToolUse` hook filters shell commands globally, but **MCP responses
bypass RTK** — they travel over stdio straight to the model. The limits are
therefore built into the tools themselves: trimmed fragments, an aggregated
ingestion report (at most 10 errors) and a capped `num_predict`.

`.rtk/filters.toml` adds project-local filters for `ollama pull`, `podman` /
`systemctl --user` (RTK knows `docker`, but not `podman`), `bigos.ingest` and
`setup.sh`. Activate them with `rtk trust` in the project directory (or
`./setup.sh --with-rtk-filters`). Nothing here depends on RTK being installed.

## Layout

```
containers/qdrant.container   Quadlet source (copied into ~/.config)
systemd/                      drop folder watcher units (path + service)
scripts/                      healthcheck, smoke test, pull_models, watcher
src/bigos/                    config, errors, ollama_client, qdrant_store,
                              loaders, ebooks, ocr, chunking, language,
                              ingest, mcp_server
data/samples/                 two PL/EN documents for the cross-lingual test
tests/                        offline tests (chunking, language, loaders)
```

## License

GNU General Public License, version 2 only. The full text is in
[LICENSE](LICENSE), and every source file carries an
`SPDX-License-Identifier: GPL-2.0-only` header. To relicense as "version 2 or
later", change those headers to `GPL-2.0-or-later`.

## Notes

- The `:Z` flag on the Quadlet volumes is kept for portability, but this host
  does not run SELinux, so there it is effectively a no-op.
- Qdrant listens on `127.0.0.1` only.
- DRM-protected books are not supported.
- No cloud service sits in the data path. The network is used only once, at
  install time: the Qdrant image, the Ollama models and the tokenizer file.