Bigos — a local RAG that eats whatever you throw in the pot

Bilingual Index Generator for Offline Search. Also a Polish stew where everything ends up in one pot, which is exactly how this works: PDFs, ebooks, Yocto recipes, C headers, Polish, English — one collection, no language filter. Like the dish, it is better the second day, once the cache is warm.

A fully local retrieval index over documents and source trees. Claude acts as the orchestrator and the data never leaves the machine: embeddings go to Ollama running natively on the host, vectors live in Qdrant inside a rootless container, and Claude talks to it through an MCP server over stdio.

The server does not generate anything. It tokenises, embeds and retrieves; the reasoning happens in the orchestrating model. See Why there is no generation here.

Architecture

Claude Code ──stdio/JSON-RPC──▶ bigos MCP server (venv, host)
                                    │
                    ┌───────────────┴───────────────┐
                    ▼                               ▼
        Ollama (native, host)              Qdrant (rootless Podman)
        └─ bge-m3 → GPU, 4 GB VRAM         127.0.0.1:6333
           keep_alive: -1                  collection: bigos

One model, one job. bge-m3 is loaded into the 4 GB of VRAM once and never evicted (keep_alive: -1); nothing else on this host asks the GPU for anything. There are no generative models in the data path at all.

Requirements

Installation

./setup.sh                      # full setup, asks before large downloads
./setup.sh --check              # diagnostics only, changes nothing
./setup.sh --yes --skip-models  # without the ~11 GB of models

Useful flags: --with-ocr (tesseract language data via sudo pacman), --with-ollama-dropin (systemd drop-in with OLLAMA_KEEP_ALIVE=-1), --with-deepseek (also pull deepseek-coder-v2:16b), --enable-linger (start Qdrant without a logged-in session), --with-rtk-filters (rtk trust).

The script is idempotent — run it as many times as you like.

Usage

Indexing a corpus

.venv/bin/python -m bigos.ingest ~/documents --recursive
.venv/bin/python -m bigos.ingest ~/ebooks/scan.pdf --ocr always
.venv/bin/python -m bigos.ingest ~/notes --include '*.md' --dry-run

Files whose content has not changed since the last run are skipped by content hash, so re-scanning a large folder costs a read per file rather than a full re-embedding. --reindex forces the work anyway; --prune also forgets documents under that path whose source file has been deleted.

Removing things from the index

The chunk text lives in the Qdrant payload, so the index is a readable copy of whatever you fed it. Removing a document from the index is what actually takes its content off this machine's search surface.

.venv/bin/python -m bigos.ingest ~/bigos-inbox/secret.pdf --forget   # one file
.venv/bin/python -m bigos.ingest ~/projects/client-x --forget        # a whole tree
curl -X DELETE http://127.0.0.1:6333/collections/bigos               # the lot

--forget works whether or not the source file still exists. Where the data sits on disk:

Nothing is written anywhere else: no ~/.cache/huggingface, no temp files that outlive a run, nothing outside the paths above.

The drop folder

Install the watcher once:

./setup.sh --with-inbox-watcher                 # default folder: ~/bigos-inbox
./setup.sh --with-inbox-watcher --inbox ~/books # or point it anywhere

From then on, dropping a file into that folder indexes it a few seconds later, and deleting a file from it removes it from the knowledge base. Two systemd user units do the work: bigos-ingest.path watches the folder with inotify IN_CLOSE_WRITE (so a half-copied file cannot trigger a run), and bigos-ingest.service waits until the folder stops changing before ingesting.

journalctl --user -u bigos-ingest -f      # watch it work
systemctl --user start bigos-ingest       # run it by hand
systemctl --user disable --now bigos-ingest.path

The watch is not recursive: creating a file deep inside a subdirectory does not trigger a run on its own, though the ingestion itself walks subdirectories, so the next trigger picks it up.

Supported documents: PDF (with OCR for scans), Markdown, TXT, EPUB, MOBI/AZW3/AZW/PRC, HTML.

Source trees

.venv/bin/python -m bigos.ingest ~/yocto/meta-mylayer --recursive
.venv/bin/python -m bigos.ingest ~/src/firmware --exclude vendor --exclude 3rdparty

Indexed as code: C/C++ (.c .h .cpp .cc .cxx .hpp .hh), Python, Bash/Zsh, BitBake/Yocto (.bb .bbappend .bbclass .inc), Make, CMake, Meson, Rust, Go, Verilog/SystemVerilog/VHDL, device tree (.dts .dtsi .dtso), assembly (.S .s), linker scripts (.lds), reST docs (.rst), YAML/TOML/INI, plus build files with no extension (Makefile, CMakeLists.txt, Dockerfile, Kconfig, Kbuild) and the ones only a glob catches: Kconfig.debug, Makefile.lib, Kbuild.include, imx8mm_evk_defconfig.

Kernel and bootloader trees

# A subsystem, not the whole tree
.venv/bin/python -m bigos.ingest ~/linux/drivers/spi --recursive
.venv/bin/python -m bigos.ingest ~/u-boot/board/freescale --recursive
.venv/bin/python -m bigos.ingest ~/linux/Documentation/spi --recursive

Verified against real sources: a Polish question about clock dividers returns spi-imx.c:1348-1398, an English one about arm64 exception vectors returns entry.S:1-46, and .rst documentation, Kconfig and _defconfig files are all searchable alongside the C.

Scope the path. A full Linux checkout is ~80k indexable files and millions of chunks - days of embedding on a laptop GPU. --max-files (default 5000) refuses to start above that and says so, rather than quietly running until Tuesday. Run --dry-run first: it reports the chunk count and an estimated embedding time before you commit.

Code is tokenised and chunked exactly like prose - there is no parser and no AST. What you get back is the file and the line range, which is what makes a hit actionable:

0.5878 [code/bitbake] nativesdk-mingw-w64-runtime_9.0.0.bb:1-28

Natural-language detection is skipped for code, because running a PL/EN detector over C or BitBake produces noise rather than metadata.

Pruning matters more than anything else here. The walk skips .git, __pycache__, node_modules, .venv, dist, build, target and the Yocto build output (tmp, tmp-glibc, sstate-cache, downloads, deploy, buildhistory) by pruning the directory, not by filtering afterwards. Add your own with --exclude. Files above 1 MB and anything with a NUL byte in its first 8 KB are skipped as generated or binary.

A word of warning learned the hard way: do not guess directory names. workdir looks like build output and in at least one real project holds the layer checkouts - excluding it silently dropped 99% of the corpus.

Re-indexing the same file overwrites its chunks instead of duplicating them: point IDs are deterministic, and a document's previous chunks are deleted before the new ones are written.

MCP tools

Restart Claude Code in this directory and the bigos server from .mcp.json connects automatically (check with /mcp).

kind narrows a search to "code" or "document". It is the one filter that exists: prose and source code answer different questions, while natural language is deliberately never filtered on.

search_knowledge_base returns raw fragments (trimmed to 800 characters each) with their provenance: file path, page number for PDFs, line range for code, heading trail for structured documents.

Why there is no generation here

An earlier version had analyze_code and generate_code running a local model on the CPU. Measured on this host (i5-12500H), asked to write an FSM in SystemVerilog with the project's own Intel FPGA guide retrieved into the prompt:

The stronger model fixed every conformance defect and none of the design ones: its 8-floor elevator had no current-floor register, and its 8-bit counter never used its direction input, so COUNT_DOWN was unreachable.

That is the whole argument. Retrieval is where a local machine wins - it is fast, private and exact. Generation over a whole module is where it loses, and routing it through a weaker model added nothing that the orchestrating model does not do better. So the server retrieves, and only retrieves.

The lesson: a 4-bit quantised 7B model holds up for short, local answers and falls apart over a whole module, even when retrieval hands it the exact coding guideline it should follow. For modules, either let the orchestrating model write the code from the retrieved sources, or switch to a stronger local model (deepseek-coder-v2:16b is a MoE with ~2.4B active parameters, so it stays usable on a CPU).

How cross-lingual search works

The whole corpus, whatever its language, goes into one collection and one vector space of the multilingual bge-m3 model. The detected language is stored in every chunk's metadata, but no query ever filters on it. That is what makes a Polish question return English fragments and the other way round. It is the entire mechanism — there is no query translation and no per-language index.

Chunking

A 512-token window with a 64-token overlap, measured with the bge-m3 tokenizer (models/tokenizer/bge-m3-tokenizer.json, downloaded once by setup.sh, fully offline afterwards). Token boundaries are mapped back to character offsets, so what lands in Qdrant is the original text, not a detokenised reconstruction.

Polish text breaks into noticeably more tokens than English of the same character length — which is why we count tokens rather than characters.

Chunk metadata: kind (document or code), code_language (c, python, bitbake, …), language (document level, prose only), chunk_language (the fragment's own language, which differs in bilingual books), source_path, source_name, page, page_end, line_start, line_end, heading_path, chunk_index, char_start, char_end, token_count, content_hash, ingested_at.

OCR

ocrmypdf is not packaged for Arch, and the pipeline needs text rather than a searchable PDF anyway. So pages without a text layer are rendered with pdftoppm (300 dpi) and recognised with tesseract (pol+eng), four pages at a time. Results are cached under ~/.cache/bigos/ocr, keyed by the file's content hash, so re-ingesting the same scan is instant.

sudo pacman -S poppler tesseract tesseract-data-pol tesseract-data-eng

Policies: auto (default — only pages without text), never, always.

Keeping the GPU in shape

nvidia-smi                      # bge-m3 should occupy ~1.3 GB of VRAM
ollama ps                       # which model sits on GPU vs CPU, and for how long
journalctl -u ollama -n 50      # which model was loaded

Every embedding request sends keep_alive: -1, so the model stays in VRAM. The systemd drop-in (--with-ollama-dropin) is optional — it only matters right after a restart of ollama.service, before the first request loads the model.

bge-m3 is the only model this project needs; ./scripts/pull_models.sh pulls just that (~1.2 GB).

Diagnostics

.venv/bin/python scripts/healthcheck.py           # status table
.venv/bin/python scripts/healthcheck.py --warmup  # + load bge-m3 into VRAM
.venv/bin/python scripts/smoke_test.py            # cross-lingual test
.venv/bin/python -m pytest tests/ -q              # offline tests
systemctl --user status qdrant
journalctl --user -u qdrant -n 50

An unreachable Qdrant or Ollama never ends in a traceback: both the MCP tools and the CLI return a single sentence with the command that fixes it.

RTK integration

The PreToolUse hook filters shell commands globally, but MCP responses bypass RTK — they travel over stdio straight to the model. The limits are therefore built into the tools themselves: trimmed fragments, an aggregated ingestion report (at most 10 errors) and a capped num_predict.

.rtk/filters.toml adds project-local filters for ollama pull, podman / systemctl --user (RTK knows docker, but not podman), bigos.ingest and setup.sh. Activate them with rtk trust in the project directory (or ./setup.sh --with-rtk-filters). Nothing here depends on RTK being installed.

Layout

containers/qdrant.container   Quadlet source (copied into ~/.config)
systemd/                      drop folder watcher units (path + service)
scripts/                      healthcheck, smoke test, pull_models, watcher
src/bigos/                    config, errors, ollama_client, qdrant_store,
                              loaders, ebooks, ocr, chunking, language,
                              ingest, mcp_server
data/samples/                 two PL/EN documents for the cross-lingual test
tests/                        offline tests (chunking, language, loaders)

License

GNU General Public License, version 2 only. The full text is in LICENSE, and every source file carries an SPDX-License-Identifier: GPL-2.0-only header. To relicense as "version 2 or later", change those headers to GPL-2.0-or-later.

Notes

• The :Z flag on the Quadlet volumes is kept for portability, but this host does not run SELinux, so there it is effectively a no-op.

• Qdrant listens on 127.0.0.1 only.

• DRM-protected books are not supported.

• No cloud service sits in the data path. The network is used only once, at install time: the Qdrant image, the Ollama models and the tokenizer file.