Hybrid RAG Project

A generalized Retrieval-Augmented Generation (RAG) system with hybrid search capabilities that works with any documents you provide. Combines semantic (dense vector) search and keyword (sparse BM25) search for optimal document retrieval, with an MCP server API for easy integration.

🎯 Key Features: Multi-format support • Local LLM • Claude Desktop integration • Structured data queries • Document-type-aware retrieval

🚀 Quick Start (No MCP Required!)

You don't need Claude Desktop or MCP to use this project! Just run:

That's it! Ask questions about the 43,835 document chunks in the sample dataset.

📖 See for complete usage instructions. 📚 Browse all documentation in the docs/ folder or start with docs/README.md.

Overview

This project implements a hybrid RAG system that combines:

• Semantic Search: Dense vector embeddings for understanding meaning and context

• Keyword Search: BM25 sparse retrieval for exact keyword matching

• Hybrid Fusion: Reciprocal Rank Fusion (RRF) to combine results from both methods

• MCP Server: Both REST API and Model Context Protocol server for Claude integration

• Multi-format Support: Automatically loads documents from various file formats

The hybrid approach ensures better retrieval accuracy by leveraging the strengths of both search methods.

Features

• Vector-based semantic search using Chroma and Ollama embeddings

• BM25 keyword search for exact term matching

• Ensemble retriever with Reciprocal Rank Fusion (RRF)

• Integration with local Ollama LLM for answer generation

• Support for multiple document formats (TXT, PDF, MD, DOCX, CSV)

• Automated document loading from data directory

• RESTful API server with /ingest and /query endpoints

• Model Context Protocol (MCP) server for Claude Desktop/API integration

• Configuration-driven architecture (no hardcoded values)

• Persistent vector store for faster subsequent queries

Architecture

Prerequisites

1. Python 3.9+

2. Ollama installed and running locally

3. Required Ollama models:

   • llama3.1:latest (or another LLM model)

   • nomic-embed-text (or another embedding model)

Installing Ollama

Visit ollama.ai to download and install Ollama for your platform.

After installation, pull the required models:

Verify Ollama is running:

Installation

1. Clone the repository:

2. Create a virtual environment:

3. Install dependencies:

Project Structure

Sample Data (UCSC Extension Project)

This repository includes 13 sample data files for demonstration and testing purposes. These files represent a realistic business scenario for TechVision Electronics and are designed to showcase the system's capabilities across multiple document types.

📊 Included Sample Files

Structured Data (CSV) - 7 files:

• product_catalog.csv - Product inventory with specifications (5,000 rows)

• inventory_levels.csv - Stock levels and warehouse data (10,000 rows)

• sales_orders_november.csv - Monthly sales transactions (8,000 rows)

• warranty_claims_q4.csv - Customer warranty claims (3,000 rows)

• production_schedule_dec2024.csv - Manufacturing schedule (4,000 rows)

• supplier_pricing.csv - Vendor pricing information (6,000 rows)

• shipping_manifests.csv - Shipping and logistics data (5,000 rows)

Unstructured Data (Markdown) - 5 files:

• customer_feedback_q4_2024.md - Customer reviews and feedback (600 chunks)

• market_analysis_2024.md - Market research and trends (400 chunks)

• quality_control_report_nov2024.md - QC findings and issues (501 chunks)

• return_policy_procedures.md - Policy documentation (300 chunks)

• support_tickets_summary.md - Technical support summary (700 chunks)

Text Data - 1 file:

• product_specifications.txt - Technical specifications (334 chunks)

Total Dataset:

• 41,000 CSV rows (chunked into 41,000 documents at 10 rows per chunk)

• 2,835 text/markdown chunks (chunked at 1000 chars with 200 char overlap)

• 43,835 total searchable document chunks

🎯 Purpose

These sample files are included to:

1. Demonstrate the system's hybrid search capabilities

2. Test both semantic (vector) and lexical (keyword) retrieval

3. Validate document-type-aware retrieval architecture

4. Provide immediate working examples without additional setup

5. Showcase cross-document query synthesis

📖 Testing Results

Comprehensive testing results are documented in TESTING_RESULTS.md, showing:

• ✅ 100% retrieval success rate across all document types

• ✅ 17 test queries with detailed results

• ✅ Performance metrics and comparative analysis

• ✅ Semantic vs Lexical vs Hybrid search comparison

💡 Using the Sample Data

Quick Start:

For Production Use: To use your own data instead:

1. Remove or backup the sample files from data/

2. Add your own documents (TXT, PDF, MD, DOCX, CSV)

3. Re-run ingestion

4. Optionally uncomment data exclusions in .gitignore

Modify this file to:

• Use different Ollama models

• Change the data directory location

• Adjust retrieval parameters (k values)

• Configure server host/port

• Change vector store persistence location

Usage

Option 1: Command Line Script

1. Add your documents to the data/ directory:

2. Run the script:

The script will:

• Load all supported documents from the data/ directory

• Initialize Ollama embeddings and LLM

• Create vector and BM25 retrievers

• Build the hybrid RAG chain

• Execute sample queries and display results

Option 2: REST API Server

1. Start the REST API server:

The server will start on http://localhost:8000

To stop the server: Press Ctrl+C for graceful shutdown

2. Ingest documents (do this first):

Response:

3. Query documents:

Response:

4. Check server status:

API Endpoints

Option 3: Claude Desktop/API via MCP

The MCP (Model Context Protocol) server allows Claude to directly query your local RAG system.

Setup for Claude Desktop

1. First, add documents to your data directory:

2. Edit the to use the correct absolute path:

3. Add this configuration to Claude Desktop:

   On macOS:

   # Copy the configuration mkdir -p ~/Library/Application\ Support/Claude # Edit the file and add your MCP server configuration nano ~/Library/Application\ Support/Claude/claude_desktop_config.json

   On Windows:

   %APPDATA%\Claude\claude_desktop_config.json

   On Linux:

   ~/.config/Claude/claude_desktop_config.json

4. Restart Claude Desktop

5. In Claude Desktop, you'll now see the MCP tools available. You can ask Claude:

   • "Use the ingest_documents tool to load my documents"

   • "Query my documents about [your question]"

   • "Check the status of the RAG system"

Available MCP Tools

Claude will have access to these tools:

Document Ingestion & Search:

• ingest_documents: Start loading and indexing documents asynchronously from the data/ directory

• get_ingestion_status: Monitor the progress of document ingestion (percentage, current file, stage)

• query_documents: Query the documents using hybrid search (semantic + keyword)

• get_status: Check the RAG system status

Structured Data Queries (for CSV files):

• list_datasets: List all available CSV datasets with columns and row counts

• count_by_field: Count rows where a field matches a value (e.g., "count people named Michael")

• filter_dataset: Get all rows matching field criteria (e.g., "all people from Company X")

• get_dataset_stats: Get statistics about a dataset (rows, columns, memory usage)

Async Ingestion with Progress Tracking

The ingestion process now runs asynchronously with real-time progress updates:

• Non-blocking: Ingestion runs in the background

• Progress tracking: See percentage complete (0-100%)

• File-level updates: Know which file is currently being processed

• Stage information: Loading files (0-80%) → Building index (80-100%) → Completed

• Status monitoring: Check progress at any time with get_ingestion_status

Example Usage with Claude

Structured Data Queries

For CSV files, use structured query tools for exact counts and filtering:

When to use each approach:

• Structured queries (count_by_field, filter_dataset): For exact counts, filtering, and structured data

• Semantic search (query_documents): For conceptual questions, understanding content, summarization

Supported File Formats

The system automatically loads and processes these formats:

• .txt - Plain text files

• .pdf - PDF documents

• .md - Markdown files

• .docx - Microsoft Word documents

• .csv - CSV files

Simply drop any supported files into the data/ directory!

How It Works

Document Loading

The DocumentLoaderUtility class:

1. Scans the data/ directory recursively

2. Identifies supported file formats

3. Uses appropriate loaders for each format

4. Adds metadata (source file, file type) to each document

5. Returns a list of Document objects ready for indexing

Hybrid Retrieval

The EnsembleRetriever uses Reciprocal Rank Fusion (RRF) to:

1. Retrieve top-k results from vector search (semantic)

2. Retrieve top-k results from BM25 search (keyword)

3. Assign reciprocal rank scores to each result

4. Combine scores to produce a unified ranking

5. Return the most relevant documents overall

This approach handles:

• Semantic queries ("How do I request time off?")

• Keyword queries ("PTO form HR-42")

• Complex queries benefiting from both methods

Customization

Using Different Models

Edit config/config.yaml to change models:

Adjusting Retrieval Parameters

Modify the k values in config/config.yaml:

Adding More File Format Support

Edit src/hybrid_rag/document_loader.py to add more loaders:

Customizing the Prompt

Edit the prompt template in scripts/run_demo.py or scripts/mcp_server.py:

Development Workflow

1. Add documents to data/ directory

2. Modify configuration in config/config.yaml as needed

3. Test with command line: python scripts/run_demo.py

4. Deploy MCP server: python scripts/mcp_server.py

5. Integrate via API in your applications

Troubleshooting

"Error connecting to Ollama"

• Ensure Ollama is installed and running

• Check that the Ollama service is accessible at the configured URL

• Verify models are downloaded: ollama list

"No documents found in data directory"

• Add files to the data/ directory

• Ensure files have supported extensions (.txt, .pdf, .md, .docx, .csv)

• Check the config/config.yaml data directory path is correct

"ModuleNotFoundError"

• Ensure virtual environment is activated: source .venv/bin/activate

• Reinstall dependencies: pip install -r requirements.txt

Poor Retrieval Results

• Add more relevant documents to the data/ directory

• Adjust k values in config/config.yaml

• Try different embedding models

• Ensure query terminology matches document content

API Errors

• Ensure you call /ingest before /query

• Check server logs for detailed error messages

• Verify Ollama is running and accessible

• Check that documents were successfully loaded

Example: Complete Workflow

Dependencies

Core libraries:

• langchain: Framework for LLM applications

• langchain-community: Community integrations

• langchain-ollama: Ollama integration

• chromadb: Vector database for embeddings

• rank-bm25: BM25 implementation for keyword search

• fastapi: Web framework for API

• uvicorn: ASGI server

• pyyaml: YAML configuration parsing

Document loaders:

• pypdf: PDF processing

• python-docx: Word document processing

• unstructured: Markdown and other formats

Performance Tips

1. Vector Store Persistence: The vector store is persisted to disk (chroma_db/) after ingestion, making subsequent queries faster.

2. Batch Processing: When adding many documents, use the /ingest endpoint once rather than multiple times.

3. Retrieval Parameters: Lower k values (e.g., 2-3) are faster and often sufficient for small document sets.

4. Model Selection: Smaller embedding models are faster but may sacrifice some accuracy.

License

This project is provided as-is for educational and demonstration purposes.

Contributing

Feel free to submit issues, fork the repository, and create pull requests for any improvements.

Resources

• LangChain Documentation

• Ollama Documentation

• ChromaDB Documentation

• FastAPI Documentation

• BM25 Algorithm

• Reciprocal Rank Fusion

Changelog

Version 2.0.0

• Generalized system to work with any documents

• Added data/ directory for document ingestion

• Created DocumentLoaderUtility for multi-format support

• Restructured project to follow Python best practices (src layout)

• Moved all configuration to config/ directory

• Moved all documentation to docs/ directory

• Created proper Python package structure with setup.py

• Organized scripts into scripts/ directory

• Updated all import paths and documentation

Version 1.0.0

• Initial implementation with sample HR documents

• Basic hybrid search with vector and BM25 retrievers