local-rag-mcp
Provides tools for querying a local SQLite database, enabling retrieval of structured data via the local_db_query tool.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@local-rag-mcpWhat does our documentation say about authentication?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
š Privacy-Preserving Local RAG with MCP Integration
A production-grade, fully local Retrieval-Augmented Generation (RAG) system integrated with the Model Context Protocol (MCP). Zero data ever leaves your machine.
Architecture
User CLI ā Agent Orchestrator ā MCP Client (stdio)
ā
āāāāāāāāāāāāāāāāāāāāāāāā
ā MCP Server ā
ā āāāāāāāāāāāāāāāāāā ā
ā ā vector_search ā ā
ā ā local_db_query ā ā
ā āāāāāāāāāāāāāāāāāā ā
āāāāāāāāāāāāāāāāāāāāāāāā
ā ā
Qdrant DB SQLite DB
(Docker) (employees)Related MCP server: OpenRAG MCP Server
Tech Stack
Component | Technology |
LLM Inference | Ollama (Llama 3) |
Embeddings | Ollama (nomic-embed-text, 768d) |
Vector Store | Qdrant (Docker) |
Re-ranking | Cross-Encoder (ms-marco-MiniLM-L-6-v2) |
MCP Server | FastMCP (Python SDK v1.x) |
Chunking | SemanticChunker (LangChain Experimental) |
Config | Pydantic Settings |
Logging | structlog (JSON structured) |
Prerequisites
Python 3.11+
Docker & Docker Compose
Ollama installed and running
At least 8GB RAM (16GB recommended)
Quick Start
1. Clone & Configure
git clone <repo-url>
cd local-rag-mcp
cp .env.example .env2. Start Qdrant
docker compose up -d
# Verify: open http://localhost:6333/dashboard3. Pull Ollama Models
# On Linux/macOS:
bash scripts/pull_models.sh
# On Windows (PowerShell):
ollama pull llama3
ollama pull nomic-embed-text4. Create Python Environment
python -m venv .venv
# Linux/macOS:
source .venv/bin/activate
# Windows:
.venv\Scripts\activate
pip install -r requirements.txt5. Seed the SQLite Database
python scripts/seed_database.py6. Ingest Documents
python main.py
# Inside the REPL:
> /ingest data/documents7. Run the Agent
python main.py
> What does our documentation say about authentication?
> List all employees in the Engineering department
> How many engineers earn above the department average?Project Structure
local-rag-mcp/
āāā docker-compose.yml # Qdrant container
āāā .env.example # Config template
āāā requirements.txt # Pinned deps
āāā pyproject.toml # Project metadata & tool config
ā
āāā config/
ā āāā settings.py # Centralized Pydantic config
ā
āāā data/
ā āāā documents/ # Drop PDFs/Markdown here
ā āāā sqlite/
ā āāā employees.db # Auto-seeded SQLite DB
ā
āāā scripts/
ā āāā seed_database.py # Seeds employee DB
ā āāā pull_models.sh # Ollama model helper
ā
āāā src/
ā āāā ingestion/ # PDF/MD loader, semantic chunker, embedder
ā āāā mcp_server/ # FastMCP server + vector_search + local_db_query tools
ā āāā agent/ # Agent loop + HyDE
ā āāā utils/ # Structured logging
ā
āāā tests/ # Pytest test suite
āāā main.py # CLI REPL entry pointCLI Commands
Command | Description |
| Ingest all PDFs/Markdown from directory |
| Toggle HyDE query enhancement |
| Show available commands |
| Exit the application |
Any other text | Ask the agent a question |
Configuration
All settings are controlled via .env. Key variables:
Variable | Default | Description |
|
| LLM for generation & tool calling |
|
| Embedding model |
|
| Qdrant collection name |
|
| Enable HyDE query enhancement |
|
| Number of final results after re-ranking |
Running Tests
pytest tests/ -vLicense
MIT
This server cannot be installed
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Orax0001/rag-agent-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server