Modular RAG System
by naffss-eng
README.md
# 🚀 Modular RAG System
### A Modular Retrieval-Augmented Generation (RAG) System with Adaptive Retrieval, Intent-Aware Query Processing, Semantic Query Rewriting, and MCP Server Integration
<p align="center">
</p>
---
# 📖 Overview
Modern enterprise organizations store thousands of pages of policies, reports, technical documents, and internal knowledge bases. Traditional keyword search often fails when users ask questions using natural language, paraphrased wording, or indirect references.
This project implements a **Modular Retrieval-Augmented Generation (RAG) System** capable of answering natural language questions over enterprise documents while providing grounded, citation-backed responses.
Unlike conventional RAG systems that treat every query identically, this system introduces **intent-aware routing**, allowing different categories of questions to follow specialized processing pipelines. The architecture further improves retrieval quality through an **Adaptive Retrieval Loop**, which selectively invokes a local Phi-3 query rewriter only when retrieval confidence is low.
The system follows a layered design emphasizing **retrieval quality**, **grounding**, **modularity**, **maintainability**, and **explainability**.
---
# 🎯 Project Objectives
The primary objectives of this project are:
- Build an MCP-compatible document question answering backend.
- Support multiple enterprise documents simultaneously.
- Generate grounded responses with citations.
- Reduce hallucinations through evidence validation.
- Improve semantic retrieval for paraphrased questions.
- Keep the architecture modular for future scalability.
- Maintain compatibility with local LLMs and cloud LLMs.
- Optimize retrieval quality while minimizing unnecessary latency.
---
# ✨ Key Features
## Intelligent Intent Routing
Instead of treating every user query identically, the system automatically classifies incoming questions into specialized pipelines.
Supported query categories include:
- Factual Questions
- Numeric Questions
- Boolean Questions
- Procedure Questions
- Summary Requests
- Comparison Questions
- Multi-document Queries
- Out-of-Scope Detection
---
## Adaptive Retrieval
Retrieval is no longer a single-pass process.
When confidence is high:
```
User Query
│
▼
Retrieval
│
▼
Answer
```
When confidence is low:
```
User Query
│
▼
Initial Retrieval
│
▼
Confidence Evaluation
│
▼
Query Rewriter (Phi-3)
│
▼
Second Retrieval
│
▼
Quality Comparison
│
▼
Best Retrieval Selected
```
This selective retrieval strategy improves semantic matching while avoiding unnecessary model execution.
---
## Query Rewriting
Instead of maintaining thousands of manually curated synonym mappings, the system leverages a **local Phi-3 model** to rewrite ambiguous queries into document-friendly terminology.
Example:
Original Query
```
What's the daily food budget?
```
Rewritten Query
```
What is the daily meal reimbursement limit during business travel?
```
The rewritten query is only used if retrieval quality objectively improves.
---
## Rewrite Validation
To prevent semantic drift, every rewritten query undergoes strict validation.
Validation rules include:
- Number preservation
- Currency preservation
- Percentage preservation
- Date preservation
- Entity preservation
- Acronym preservation
- Logical operator preservation
- Polarity preservation
Example:
Original
```
Can employees NOT claim cash gifts?
```
Rejected Rewrite
```
Can employees claim cash gifts?
```
The validator immediately rejects such rewrites.
---
## Rewrite Cache
Repeated rewrites are avoided using an LRU cache.
Cache Key
```
Hash(
Normalized Query
+
Document Collection Hash
)
```
Benefits
- Reduced latency
- Reduced Phi-3 execution
- Deterministic behaviour
- Automatic invalidation on document changes
---
# 🏗 System Architecture
```
┌────────────────────┐
│ User │
└─────────┬──────────┘
│
▼
┌────────────────────────┐
│ MCP Server │
└─────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Intent Detection & Routing │
└──────┬───────────┬──────────┘
│ │
▼ ▼
Standard Pipeline Summary Pipeline
│
▼
Comparison / Numeric /
Boolean / Procedure etc.
│
▼
Adaptive Retrieval Layer
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
Query Rewriter Validator Rewrite Cache
│ │ │
└──────────────┼──────────────┘
▼
Final Retrieval
│
▼
Evidence Validation
│
▼
Gemini 2.5 Flash Generator
│
▼
JSON Response + Citations
```
---
# 🧠 Layered RAG Pipeline
```
Document Upload
│
▼
Document Parsing
│
▼
Chunk Generation
│
▼
Embedding Generation
│
▼
ChromaDB Storage
│
──────────────────────────────────────────
Runtime
──────────────────────────────────────────
│
User Query
│
▼
Intent Detection
│
▼
Vector Retrieval
│
▼
Adaptive Retrieval (Optional)
│
▼
Evidence Validation
│
▼
LLM Generation
│
▼
Grounded JSON Response
```
---
# 🧩 Core Components
| Component | Responsibility |
|------------|---------------|
| MCP Server | Exposes the QA tool to MCP clients |
| Intent Router | Routes queries to specialized pipelines |
| Retriever | Retrieves relevant chunks from ChromaDB |
| Query Rewriter | Improves low-confidence semantic retrieval |
| Rewrite Validator | Prevents semantic drift |
| Rewrite Cache | Avoids repeated rewrites |
| Evidence Validator | Ensures answer grounding |
| Gemini Generator | Produces grounded final responses |
| Phi-3 | Local semantic query optimization |
---
# 📂 Project Structure
```
modular-rag-system/
│
├── data/
│ ├── company_policy.txt
│ ├── conduct_policy.txt
│ ├── finance_policy.txt
│ ├── it_policy.pdf
│ └── remote_work_agreement.pdf
│
├── src/
│ ├── answer_question.py
│ ├── mcp_server.py
│ ├── query_rewriter.py
│ ├── retrieve.py
│ ├── embed_document.py
│ ├── chunk_document.py
│ ├── read_document.py
│ ├── evidence_engine.py
│ ├── layer2_validator.py
│ ├── store_embeddings.py
│ ├── verify_storage.py
│ ├── config.py
│ ├── test_acceptance_suite.py
│ ├── test_query_rewriter.py
│ └── ...
│
├── requirements.txt
├── README.md
└── .gitignore
```
---
# ⚙ Technology Stack
| Category | Technology |
|-----------|------------|
| Language | Python |
| Vector Database | ChromaDB |
| Embedding Model | all-MiniLM-L6-v2 |
| Primary LLM | Gemini 2.5 Flash |
| Local Model | Phi-3 (Ollama) |
| Protocol | Model Context Protocol (MCP) |
| Retrieval | Dense Vector Search |
| Validation | Rule-Based Evidence Validation |
| Environment | VS Code + Python Virtual Environment |
---
# 🚀 Installation
Clone the repository
```bash
git clone https://github.com/naffss-eng/modular-rag-system.git
cd modular-rag-system
```
Create a virtual environment
```bash
python -m venv venv
```
Activate the environment
Windows
```bash
venv\Scripts\activate
```
Linux
```bash
source venv/bin/activate
```
Install dependencies
```bash
pip install -r requirements.txt
```
Create a `.env` file
```text
GOOGLE_API_KEY=YOUR_API_KEY
```
# ▶ Running the Project
## Step 1 — Ingest Documents
Load and preprocess the documents.
```bash
python src/read_document.py
```
---
## Step 2 — Chunk Documents
Generate semantic chunks.
```bash
python src/chunk_document.py
```
---
## Step 3 — Generate Embeddings
```bash
python src/embed_document.py
```
---
## Step 4 — Store Embeddings
```bash
python src/store_embeddings.py
```
---
## Step 5 — Verify Vector Database
```bash
python src/verify_storage.py
```
---
## Step 6 — Start the MCP Server
```bash
python src/mcp_server.py
```
---
## Step 7 — Connect Using Antigravity CLI
```bash
antigravity connect localhost:8000
```
Once connected, users can directly ask questions over the uploaded document collection.
---
# 💬 Example Queries
### Factual
```
What are the company's core collaboration hours?
```
---
### Numeric
```
What is the daily meal reimbursement limit?
```
---
### Boolean
```
Are cash gifts allowed?
```
---
### Procedure
```
What documents are required to submit a travel reimbursement claim?
```
---
### Summary
```
Give me an overview of the IT Policy.
```
---
### Comparison
```
Compare the Finance Policy with the Company Policy.
```
---
### Multi-document
```
What are the remote work responsibilities mentioned across all documents?
```
---
### Out-of-Scope
```
How do I bake a chocolate cake?
```
Expected behaviour:
```
Grounding Status : REJECTED
Answer:
I could not find this information in the uploaded documents.
```
---
# 📊 Supported Query Types
| Query Type | Supported |
|------------|-----------|
| Factual | ✅ |
| Numeric | ✅ |
| Boolean | ✅ |
| Procedure | ✅ |
| Summary | ✅ |
| Comparison | ✅ |
| Multi-document | ✅ |
| Out-of-Scope Detection | ✅ |
---
# 🔄 Intent-Aware Processing
Unlike traditional RAG systems that process every query through a single retrieval pipeline, this project dynamically routes each request based on its semantic intent.
```
User Query
│
▼
Intent Classification
│
┌───────────────┼────────────────┐
│ │ │
▼ ▼ ▼
Summary Comparison Standard QA
│ │ │
▼ ▼ ▼
Dedicated Balanced Dual Adaptive
Pipeline Retrieval Retrieval
```
This modular routing significantly improves maintainability while allowing future pipelines to be integrated independently.
---
# 🔍 Adaptive Retrieval Loop
One of the major architectural enhancements introduced in Backend v2.0 is the Adaptive Retrieval Loop.
Instead of rewriting every query, the system first evaluates retrieval confidence.
```
User Query
│
▼
Initial Retrieval
│
▼
Confidence Evaluation
│
┌──────────────┴──────────────┐
│ │
▼ ▼
High Confidence Low Confidence
│ │
▼ ▼
Continue Query Rewriter (Phi-3)
│
▼
Rewrite Validator
│
▼
Rewrite Cache Check
│
▼
Second Retrieval
│
▼
Quality Comparison
│
┌──────────────┴─────────────┐
▼ ▼
Better Retrieval Worse Retrieval
│ │
▼ ▼
Use New Chunks Keep Original Chunks
```
Only validated improvements are accepted, ensuring retrieval quality never degrades.
---
# 🛡 Grounding & Evidence Validation
Generating an answer is not sufficient; the answer must also be supported by retrieved evidence.
Before returning a response, the system validates whether the retrieved context actually contains sufficient information to answer the question.
Possible grounding outcomes:
| Status | Meaning |
|---------|---------|
| PASSED | Sufficient evidence exists |
| REJECTED | Retrieved evidence is insufficient |
If grounding fails, the system returns a transparent response rather than hallucinating unsupported information.
---
# 📑 JSON Response Format
The backend returns structured responses in JSON.
Example:
```json
{
"answer": "...",
"grounding_status": "PASSED",
"constraint_result": "NOT_APPLICABLE",
"citations": [
"finance_policy_3"
],
"execution_time_ms": 2943,
"initialization_time_ms": 0.01
}
```
---
# 📈 Performance Summary
The backend was evaluated using both automated benchmark suites and manual acceptance testing.
## Automated Validation
| Metric | Result |
|---------|---------|
| Benchmark Queries | 340+ |
| Regression Tests | 100% Pass |
| Acceptance Suite | 100% Pass |
| Rewrite Validator Tests | 29/29 Pass |
---
## Manual Validation
A final manual evaluation was conducted using representative enterprise queries covering:
- Factual retrieval
- Numeric reasoning
- Boolean questions
- Summary generation
- Procedure extraction
- Comparison
- Multi-document reasoning
- Out-of-scope detection
**Result**
```
15 / 15 Queries Passed
```
---
# ⚡ Performance Optimizations
Several optimizations were introduced to balance retrieval quality with runtime efficiency.
### Adaptive Query Rewriting
Only low-confidence retrievals trigger semantic rewriting.
---
### Rewrite Validation
Rejects semantic drift before retrieval.
---
### LRU Rewrite Cache
Avoids repeated Phi-3 inference for identical queries.
---
### Feature Flags
Every Backend v2.0 enhancement is protected behind configuration flags.
```
ENABLE_QUERY_REWRITER = True
```
Rollback is immediate by disabling the feature.
---
# 🧪 Testing Strategy
Testing was performed at multiple levels.
### Unit Tests
- Query Rewriter
- Rewrite Validator
- Rewrite Cache
- Adaptive Retrieval
---
### Regression Tests
- Preprocessing
- Retrieval
- Empty Queries
- Final Verification
---
### Acceptance Tests
Realistic enterprise policy questions covering every supported intent.
---
### Stress Tests
- Long queries
- Empty input
- Malformed characters
- Out-of-scope requests
- Timeout recovery
- Ollama unavailable
- Cache failures
---
# 🎯 Engineering Decisions
Several important architectural decisions shaped the final system.
### Why Modular Pipelines?
Each query type requires different retrieval and reasoning behaviour. A modular architecture allows independent optimisation of each pipeline.
---
### Why Local Phi-3?
Using Phi-3 locally preserves Gemini API quota while enabling semantic query rewriting entirely offline.
---
### Why Confidence-Based Rewriting?
Most queries already retrieve correct results.
Running an LLM for every query would unnecessarily increase latency.
Adaptive retrieval ensures rewriting occurs only when beneficial.
---
### Why Rewrite Validation?
LLMs occasionally alter numbers, entities, or logical meaning.
The Rewrite Validator prevents such semantic drift before retrieval.
---
### Why Rewrite Cache?
Repeated enterprise questions are common.
Caching validated rewrites reduces latency and eliminates redundant local inference.
---
# 🚧 Current Limitations
Current limitations include:
- Dense vector retrieval only (no hybrid BM25 retrieval)
- CPU inference latency for local Phi-3
- No web interface (backend only)
- No authentication layer
- Single-session document workspace
---
# 🛣 Future Roadmap
### Backend v2.1
- Hybrid Retrieval (Dense + BM25)
- Cross-Encoder Reranking
- Better telemetry dashboard
---
### Backend v3
- Agentic Retrieval
- Tool-calling workflows
- Multi-agent reasoning
- Knowledge graph integration
---
### Frontend
- Streamlit / React interface
- Drag-and-drop document upload
- Citation viewer
- Interactive retrieval visualization
- Chat history
---
# 🤝 Acknowledgements
This project was developed as part of an internship focused on building a modular Retrieval-Augmented Generation system using modern NLP techniques.
It combines dense retrieval, adaptive query rewriting, evidence validation, and Model Context Protocol (MCP) integration into a scalable and extensible architecture.
---
# 📜 License
This project is intended for educational and research purposes.
---
# 👩💻 Author
**Nafisa Hasan**
B.Tech Computer Science Engineering
Special interests:
- Natural Language Processing
- Retrieval-Augmented Generation
- Large Language Models
- Information Retrieval
- Applied Machine Learning
---
# ⭐ Final Remarks
This project demonstrates how a traditional Retrieval-Augmented Generation pipeline can be transformed into a modular, intent-aware architecture capable of handling diverse enterprise document question-answering tasks with improved retrieval quality, grounded responses, and extensible design.
Rather than relying on a single retrieval strategy, the system incorporates adaptive retrieval, semantic query rewriting, rule-based validation, evidence grounding, and modular routing to improve both reliability and maintainability.
The resulting architecture serves as a strong foundation for future work in hybrid retrieval, agentic workflows, and production-scale document intelligence systems.
---
> *"Good retrieval finds information. Great retrieval understands the question first."*
This server cannot be deployed
Maintenance
ActivityStale
ResponsivenessNo issues