How do I use Modular RAG MCP Server?

This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.

Modular RAG MCP Server

by CrstalLi02

Overview Schema Source Related Servers Discussions Admin

Python

Local

Modular RAG MCP Server

A pluggable, observable Retrieval-Augmented Generation (RAG) service that exposes its knowledge base as tools over the Model Context Protocol (MCP), so AI assistants such as GitHub Copilot, Claude Desktop, Claude Code and Cursor can query your private documents directly.

📖 Table of Contents

Related MCP server: rag-mcp-server

🏗️ Overview

Modular RAG MCP Server is an end-to-end RAG system covering the full lifecycle of a knowledge base:

Module	Capability	Details
Ingestion Pipeline	PDF → Markdown → Chunk → Transform → Embedding → Upsert	Full ingestion pipeline with multimodal image captioning
Hybrid Search	Dense (vector) + Sparse (BM25) + RRF fusion + Rerank	Two-stage retrieval: broad recall followed by precise re-ranking
MCP Server	Standard MCP tools	`query_knowledge_hub`, `list_collections`, `get_document_summary`
Dashboard	Six-page Streamlit admin console	System overview / data browser / ingestion management / ingestion traces / query traces / evaluation
Evaluation	Ragas + custom evaluators	Regression testing against a golden test set instead of tuning "by feel"
Observability	End-to-end tracing	Every intermediate state of both the ingestion and query pipelines is inspectable
Agent Skills	Spec-driven development workflow	`auto-coder`, `qa-tester`, `setup`, `package` and other skills cover coding, testing, setup and packaging

The complete design — architecture, module interfaces, test strategy and the phased task schedule — is documented in DEV_SPEC.md.

🧭 Architecture

                 ┌──────────────────────────────────────────────┐
  MCP Clients    │  Copilot · Claude Desktop · Claude Code · …  │
                 └───────────────────────┬──────────────────────┘
                                         │ MCP (stdio / JSON-RPC)
                 ┌───────────────────────▼──────────────────────┐
  MCP Server     │ query_knowledge_hub · list_collections ·     │
                 │ get_document_summary                         │
                 └───────────────────────┬──────────────────────┘
                                         │
                 ┌───────────────────────▼──────────────────────┐
  Query Engine   │ Query processing → Dense + BM25 retrieval →  │
                 │ RRF fusion → (optional) rerank → response    │
                 │ builder with citations                       │
                 └───────────────────────┬──────────────────────┘
                                         │
                 ┌───────────────────────▼──────────────────────┐
  Storage        │ Vector store (Chroma) · BM25 index · images  │
                 └───────────────────────▲──────────────────────┘
                                         │
                 ┌───────────────────────┴──────────────────────┐
  Ingestion      │ Load PDF → split → refine / enrich metadata →│
                 │ caption images (Vision LLM) → embed → upsert │
                 └──────────────────────────────────────────────┘

  Cross-cutting: pluggable libs (LLM / Embedding / Reranker / Splitter /
  VectorStore / Evaluator / Loader) · tracing · Streamlit dashboard

✨ Key Features

🔌 Fully pluggable architecture — LLM, embedding, reranker, splitter, vector store and evaluator each sit behind an abstract interface with a factory. Swap backends by editing config/settings.yaml; no code changes required.

🔍 Hybrid search + reranking — BM25 sparse retrieval handles exact matches on proper nouns and terminology, dense embeddings handle synonyms and semantic similarity. Results are merged with Reciprocal Rank Fusion (RRF), then optionally re-ranked by a Cross-Encoder or an LLM to balance recall and precision.

🖼️ Multimodal image handling — An image-to-text strategy uses a Vision LLM to caption images and stitches the captions into their chunks, so the plain-text RAG pipeline can retrieve images by text queries.

📡 MCP ecosystem integration — Implements the Model Context Protocol standard, so any MCP client can use the knowledge base without building a frontend. Build once, use everywhere.

📊 Visual management + automated evaluation — A Streamlit dashboard provides data management and pipeline tracing; Ragas and custom evaluators create a data-driven feedback loop.

🧪 Three-layer test suite — Unit, integration and end-to-end tests cover isolated module logic, cross-module interactions and full flows (MCP client, dashboard).

🚀 Getting Started

Prerequisites

Python 3.10+
An API key for at least one LLM / embedding provider (OpenAI, Azure OpenAI, DeepSeek), or a local Ollama install

Installation

git clone https://github.com/CrstalLi02/modular-rag-mcp-server.git
cd modular-rag-mcp-server

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

pip install -e ".[dev]"

Configure providers

Edit config/settings.yaml to choose your LLM, embedding, vision and vector-store providers, and export the matching API keys (for example OPENAI_API_KEY).

Alternatively, open the project in an AI coding assistant that supports agent skills (Claude Code, GitHub Copilot) and run the setup skill. It walks through provider selection, API key configuration, dependency installation, config generation and launching the dashboard.

🛠️ Usage

Ingest documents

# A single file
python scripts/ingest.py --path documents/report.pdf --collection contracts

# A whole directory
python scripts/ingest.py --path documents/ --collection technical_docs

# Force re-ingestion of already-processed files
python scripts/ingest.py --path documents/report.pdf --collection contracts --force

Query from the command line

python scripts/query.py --query "How do I configure Azure OpenAI?" --collection technical_docs
python scripts/query.py --query "What is RRF?" --verbose
python scripts/query.py --query "What is RRF?" --no-rerank

Run the evaluation suite

python scripts/evaluate.py
python scripts/evaluate.py --test-set path/to/golden.json --collection technical_docs

Launch the dashboard

python scripts/start_dashboard.py            # http://localhost:8501
python scripts/start_dashboard.py --port 8502

Start the MCP server

python -m src.mcp_server.server   # run from the project root

🔗 Connecting an MCP Client

The server speaks MCP over stdio, so any MCP-compatible tool can use it.

Claude Desktop / Claude Code (claude_desktop_config.json or .mcp.json):

{
  "mcpServers": {
    "knowledge-hub": {
      "command": "bash",
      "args": ["-c", "cd /absolute/path/to/modular-rag-mcp-server && .venv/bin/python -m src.mcp_server.server"]
    }
  }
}

GitHub Copilot (VS Code) — add the same server under servers in .vscode/mcp.json.

Cursor — add it under mcpServers in .cursor/mcp.json.

Exposed tools:

Tool	Description
`query_knowledge_hub`	Hybrid search over a collection; returns ranked chunks with source citations
`list_collections`	Lists available collections and their document counts
`get_document_summary`	Returns the summary and metadata of an ingested document

⚙️ Configuration

All behaviour is driven by config/settings.yaml:

Section	Options
`llm.provider`	`openai`, `azure`, `ollama`, `deepseek`
`embedding.provider`	`openai`, `azure`, `ollama`
`vision_llm.provider`	`openai`, `azure`, `ollama`
`vector_store.provider`	`chroma` (interfaces prepared for `qdrant`, `pinecone`)
`rerank.provider`	`none`, `cross_encoder`, `llm`
`evaluation.provider`	`ragas`, `deepeval`, `custom`

Prompt templates for chunk refinement, metadata enrichment, image captioning and LLM reranking live in config/prompts/.

🧪 Testing

pytest tests/unit                 # fast, no external services
pytest tests/integration          # requires provider credentials
pytest tests/e2e                  # full MCP client / dashboard flows

Integration tests read credentials from config/test_credentials.yaml (copy it from config/test_credentials.yaml.example).

📂 Project Structure

├── config/              # settings.yaml + prompt templates
├── scripts/             # ingest / query / evaluate / dashboard entry points
├── src/
│   ├── core/            # settings, shared types, query engine, response builder, tracing
│   ├── ingestion/       # chunking, transforms, embedding, storage, pipeline
│   ├── libs/            # pluggable backends: llm, embedding, reranker, splitter,
│   │                    #   vector_store, loader, evaluator
│   ├── mcp_server/      # MCP protocol handler, server and tools
│   └── observability/   # logging, evaluation, Streamlit dashboard
├── tests/               # unit / integration / e2e + fixtures
├── .claude/skills/      # agent skills (Claude Code)
├── .github/skills/      # agent skills (GitHub Copilot)
├── DEV_SPEC.md          # full design spec and task schedule
└── src/mcp_server/server.py   # MCP server entry point (stdio)

🧩 Extending the System

Add a provider (e.g. Qwen, another vector DB) — every backend in src/libs/ uses the factory pattern. To add one: (1) implement the base class, (2) register it in the factory, (3) select it in settings.yaml. Most providers follow the OpenAI request format, so new LLM backends are usually small.

Support more document formats — the loader layer is built around BaseLoader. PDF is implemented today; Word, Markdown or HTML support only needs a new loader class registered alongside the PDF loader.

📌 Roadmap

Item	Status
Custom evaluator	Framework in place; evaluation methods and golden dataset to be completed
Cross-Encoder reranker	Framework in place; needs end-to-end testing with a local model such as `cross-encoder/ms-marco-MiniLM-L-6-v2`
Additional loaders (Word / Markdown / HTML)	Planned
Qdrant / Pinecone vector stores	Planned

📄 License

Released under the MIT License.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Resources

Related MCP Connectors

DocsMintA
io.github.HiAi-gg
Cloud or self-hosted knowledge for AI agents: hybrid search, reranking, GraphRAG, scoped MCP tools.
Documentation Access Knowledge & Memory RAG Systems
22174
https://docsmint.com/mcp
ragie
com.mcparmory
Ingest, manage, and retrieve documents for RAG-powered AI applications
RAG Systems Databases Knowledge & Memory
2
https://mcp.mcparmory.com/ragie
KnowDrive
ai.knowdrive
Private, multimodal knowledge store for agents: hybrid search with cited, provenance-backed results.
RAG Systems Knowledge & Memory Search
1
https://knowdrive.ai/api/v1/mcp
Keepfor.meA
io.github.melcutz
Your personal library, readable by AI agents: durable snapshots, hybrid search, 46 MCP tools.
Knowledge & Memory RAG Systems Search
461
https://app.keepfor.me/api/mcp

Related MCP Servers

MCP Knowledge Service
RAG Systems Knowledge & Memory
hyh0620
A
license
Not graded
quality
D
maintenance
Provides hybrid retrieval (dense + BM25 + RRF) with collection-based isolation and document ingestion for private knowledge access via MCP.
Updated 4 months ago (2026-07-24 08:54 UTC)
MIT
rag-mcp-server
RAG Systems Search
q6066697
A
license
A
quality
D
maintenance
Exposes hybrid retrieval (dense embeddings + BM25 + RRF) and document operations (search, fetch, rerank) as MCP tools, using Qdrant and OpenAI embeddings for local or server mode.
Updated 3 months ago (2026-08-19 21:59 UTC)
3
MIT
Modular RAG MCP Server
Agent Orchestration AI & Machine Learning
MiemieYang-yang
A
license
Not graded
quality
D
maintenance
Enables AI assistants to perform retrieval-augmented generation with hybrid search, reranking, multi-modal image processing, and RAG evaluation through standardized MCP tools.
Updated 3 months ago (2026-08-21 09:55 UTC)
MIT
Modular RAG MCP Server
RAG Systems Search
YiYunn-lucky
A
license
Not graded
quality
C
maintenance
Enables AI assistants to perform retrieval-augmented generation through MCP tools for querying knowledge hubs, listing collections, and retrieving document summaries, with pluggable ingestion, hybrid search, and evaluation components.
Updated 2 months ago (2026-09-07 06:15 UTC)
MIT

Modular RAG MCP Server

📖 Table of Contents

🏗️ Overview

🧭 Architecture

✨ Key Features

🚀 Getting Started

Prerequisites

Installation

Configure providers

🛠️ Usage

🔗 Connecting an MCP Client

⚙️ Configuration

🧪 Testing

📂 Project Structure

🧩 Extending the System

📌 Roadmap

📄 License

Tool Schema Changelog

Resources

Related MCP Connectors

Related MCP Servers

MCP Knowledge Service

rag-mcp-server

Modular RAG MCP Server

Modular RAG MCP Server