RAG System

Local Retrieval-Augmented Generation system for PDF documentation. Provides offline document ingestion with hybrid vector search and an MCP server for LLM agents to query indexed documentation.

Architecture

                 Offline / manual
PDFs and docs -> Ingestion CLI -> Local Qdrant (Docker)
                                      |
                 During questions     |
LLM agent -> MCP retrieval tools -----+
          -> grounded answer with citations

Two executables share a common codebase:

• rag - Ingestion CLI for parsing, chunking, embedding, and storing PDFs

• rag-mcp - Read-only MCP stdio server exposing search and retrieval tools

Related MCP server: okfy

Prerequisites

• Python 3.11+

• uv package manager

• Docker (for Qdrant)

Setup

1. Start Qdrant

cd rag-system
docker compose up -d

Qdrant will be available at http://localhost:6333.

2. Install Python dependencies

uv sync

This creates a virtual environment and installs all dependencies including local embedding models.

3. Copy environment configuration

cp .env.example .env

Edit .env if you need to change Qdrant URL, collection name, or model settings.

Ingesting Documents

Place PDF files in the documents/ directory (or reference them by path).

# Ingest a single file
uv run rag ingest ./documents/manual.pdf

# Ingest with a custom document ID
uv run rag ingest ./documents/manual.pdf --id product-manual

# Force re-ingestion (replaces existing version)
uv run rag ingest ./documents/manual.pdf --force

# Ingest all PDFs in a directory
uv run rag ingest-directory ./documents

# List indexed documents
uv run rag list

# Inspect a document's metadata and sample chunks
uv run rag inspect product-manual

# Delete a document
uv run rag delete product-manual

Embedding models (BAAI/bge-small-en-v1.5 for dense, Qdrant/bm25 for sparse) are downloaded on first use and cached locally.

MCP Server Configuration

Add the RAG MCP server to your agent's MCP configuration:

opencode (opencode.json or ~/.config/opencode/opencode.json)

{
  "mcp": {
    "servers": {
      "rag-documentation": {
        "type": "local",
        "command": ["uv", "run", "--directory", "/absolute/path/to/rag-system", "rag-mcp"],
        "enabled": true
      }
    }
  }
}

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "rag-documentation": {
      "command": "uv",
      "args": ["run", "--directory", "/absolute/path/to/rag-system", "rag-mcp"]
    }
  }
}

MCP Tools

The server exposes four read-only tools:

Configuration

All settings are read from environment variables (or .env file):

Development

# Run tests (unit tests run without Qdrant)
uv run pytest tests/unit

# Run all tests (requires Qdrant running)
docker compose up -d
uv run pytest

# Lint
uv run ruff check src tests

# Type check
uv run mypy src

How Retrieval Works

1. Hybrid search: Dense vector search (semantic similarity) and sparse BM25 search (keyword matching) run in parallel against Qdrant.

2. Reciprocal Rank Fusion (RRF): Qdrant fuses dense and sparse results using RRF.

3. Citations: Every result includes the document title, section path, and page range.

4. Context expansion: The get_document_context tool retrieves neighboring chunks when a search result needs more surrounding text.

All processing is local. No document content is sent to external services.